Semantic-guided face inpainting with subspace pyramid aggregation

Abstract

With the recent advancement of Generative Adversarial Networks, image inpainting has been improved, but the complexity of face structure makes face inpainting more challenging. The main reasons are attributed to two points: (1) the lack of geometry relation between facial features to synthesize fine textures, and (2) the difficulty of repairing occluded area based on known pixels at a distance, especially when the face is occluded over a large area. This paper proposes a face inpainting method based on semantic feature guidance and aggregated subspace pyramid module, where we use the semantic features of masked faces as the prior knowledge to guide the inpainting of masked areas. Besides, we propose an ASPM (Aggregated Subspace Pyramid Module), which aggregates contextual information from different receptive fields and allows the of capturing distant information. We do experiments on the CelebAMask-HQ dataset and the FlickrFaces-HQ dataset, qualitative and quantitative studies show that it surpasses state-of-the-art methods. Code is available at https://github.com/xiumin123/Face_ inpainting.