Dr. Guilin Liu
NVIDIA Senior Research Scientist
Partial Convolution for Image Inpainting
Virtual – 11:00 am EST – Microsoft Teams meeting – Join here(link is external)
Existing deep learning-based image inpainting methods use a standard convolutional network over the corrupted image, usi g convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value). This often leads to artifacts such as color discrepancy and blurriness. Postprocessing is usually used to reduce such artifacts but is expensive and may fail. In this talk, I will introduce partial convolution, where the convolution is masked and renormalized to be conditioned on only valid pixels. I will further include a mechanism to automatically generate an updated mask for the next layer as part of the forward pass. The image inpainting model with partial convolution achieves both better qualitative and quantitative than other methods. Besides the image inpainting task, the partial convolution idea can further be extended to serve as a new padding scheme for general convolution layer. This is based on the intuition that the padded region can be treated as holes and the original input as non-holes. I will compare this partial convolution-based padding with existing paddings on various computer vision tasks. I will also show how partial convolution can be used for instance-based image synthesis.
Guilin Liu is a senior research scientist at NVIDIA. He obtained his Ph.D. in Computer Science from George Mason University in 2017. He received his B.E. in Spatial Informatics and Digitalized Technology together with a minor degree in Finance from Wuhan University in 2012. He was a research intern at TTI Chicago in 2015 summer and at Adobe Research in 2016 summer. His research has attracted many attentions from some media outlets like Forbes, MIT Technology Review, Yahoo Finance etc. Two of his research videos have attracted more than one million views on YouTube. His current research interests lie in using deep learning for image/video editing and creation, rendering, geometric data processing, 3D reconstruction and beyond. More information can be found at: https://liuguilin1225.github.io/(link is external)
IS&T Colloquium Committee Host: Dr. Daniel Duffy(link sends e-mail)
Sign language interpreter upon request: 301-286-7348
Request future announcements: ist-colloq-subscribe@lists.nasa.gov(link sends e-mail)