An adaptive contextual learning network for image inpainting

Abstract

Deep-learning-based methods for image inpainting have been intensively researched because of deep neural networks’ powerful approximation capabilities. In particular, the context-reasoning-based methods have shown significant success. Nonetheless, images generated using these methods tend to suffer from visually inappropriate content. This is due to the fact that their context reasoning processes are weakly adaptive, limiting the flexibility of generation. To this end, this paper presents an adaptive contextual learning network (ACLNet) for image inpainting. The main contribution of the proposed method is to significantly improve the adaptive capability of the context reasoning. The method can adaptively weigh the importance of known contexts for filling missing regions, ensuring that the filled content is finely filtered rather than raw, which improves the reliability of the generated content. Specifically, a modular hybrid dilated residual unit and an adaptive region affinity learning attention are created, which can adaptively choose and aggregate contexts based on the sample itself through gating mechanism and similarity filtering respectively. The extensive comparisons reveal that ACLNet exceeds the state-of-the-art, improving peak signal-to-noise ratio (PSNR) by 0.25 dB and structural similarity index measure (SSIM) by 0.017 on average and that it can generate more aesthetically realistic images than other approaches. The implemented ablation experiments also confirm the effectiveness of ACLNet.