Learning region-aware style-content feature transformations for face image beautification

Abstract

As a representative image-to-image translation task, facial makeup transfer is typically performed by applying intermediate feature normalization, conditioned on the style information extracted from a reference image. However, the relevant methods are typically limited in range of applicability, due to that the style information is independent of source images and lack of spatial details. To realize precise makeup transfer and further associate with face component editing, we propose a Semantic Region Style-content Feature Transformation approach, which is referred to as SRSFT. Specifically, we encode both reference and source images into region-wise feature vectors and maps, based on semantic segmentation masks. To address the misalignment in poses and expressions, region-wise spatial transformations are inferred to align the reference and source masks, and are then applied to explicitly warp the reference feature maps to the source face, without any extra supervision. The resulting feature maps are fused with the source ones and inserted into a generator for image synthesis. On the other hand, the reference and source feature vectors are also fused and used to determine the modulation parameters at multiple intermediate layers. SRSFT is able to achieve superior beautification performance in terms of seamlessness and fidelity.