Deep image inpainting with region prediction at hierarchical scales

Abstract

In this paper, we propose a CNN based method for image inpainting, which utilizes the inpaintings generated at different hierarchical resolutions. Firstly, we begin with the prediction of the missing image region with larger contextual information at the lowest resolution using deconv layers. Next, we refine the predicted region at greater hierarchical scales by imposing gradually reduced contextual information surrounding the predicted region by training different CNNs. Thus, our method not only utilizes information from different hierarchical resolutions but also intelligently leverages the context information at different hierarchy to produce better inpainted image. The individual models are trained jointly, using loss functions placed at intermediate layers. Finally, the CNN generated image region is sharpened using the unsharp masking operation, followed by intensity matching with the contextual region, to produce visually consistent and appealing inpaintings with more prominent edges. Comparison of our method with well-known inpainting methods, on the Caltech 101 objects dataset, demonstrates the quantitative and qualitative strengths of our method over the others.