FaST-Net: Face Style Self-Transfer Network for masked face inpainting

Abstract

During the COVID-19 coronavirus epidemic, people usually wear masks to prevent the spread of the virus, which has become a major obstacle when we use face-based computer vision techniques such as face recognition and face detection. So masked face inpainting technique is desired. Actually, the distribution of face features is strongly correlated with each other, but existing inpainting methods typically ignore the relationship between face feature distributions. To address this issue, in this paper, we first show that the face image inpainting task can be seen as a distribution alignment between face features in damaged and valid regions, and style transfer is a distribution alignment process. Based on this theory, we propose a novel face inpainting model considering the probability distribution between face features, namely Face Style Self-Transfer Network (FaST-Net). Through the proposed style self-transfer mechanism, FaST-Net can align the style distribution of features in the inpainting region with the style distribution of features in the valid region of a face. Ablation studies have validated the effectiveness of FaST-Net, and experimental results on two popular human face datasets (CelebA and VGGFace) exhibit its superior performance compared with existing state-of-the-art methods.