Empirical Study of U-Net Based Models for 2D Face Recovery from Face-Mask Occlusions

Human face related digital technologies have been widely applied in various fields including face recognition based biometrics, facial landmarks based face deformation for gaming, facial reconstruction for those who are disfigured from an accident in the medical field and others. Such technologies typically rely on the information of a full, uncovered face and their performance would suffer varying degrees of deterioration according to the level of facial occlusion exhibited. 2D face recovery from occluded faces has therefore become an important research area as it is both crucial and desirable to attain full facial information before it is used in downstream tasks. In this paper, we address the problem of 2D face recovery from facial-mask occlusions, a pertinent issue that is widely observed in situations such as the Covid-19 pandemic. In recent trends, most researches are carried out through deep learning techniques to recover masked faces. The whole process consists of two tasks which are image segmentation and image inpainting. As U-Net is a typical deep learning model for image segmentation, but it also helpful in image inpainting and image colorization, so it has been frequently used in solving face recovery problems. To further explore the capability of U-Net and its variants for face recovery from masked faces, we propose to conduct a comparative study on several U-Net based models on a synthetic dataset that was generated based on public face datasets and mask generator. Results showed that Resnet U-Net and VGG16 U-Net had performed better in face recovery among the six different U-Net based models.