Controllable Facial Micro-element Synthesis using Segmentation Maps

Abstract

In facial image synthesis, the style of the source image is converted using a reference image, or images with different styles are synthesized by each attribute using a facial attribute segmentation map. However, previous works cannot deal with the fine areas because the style is changed mostly in large areas such as hair, eyes, and mouth. To overcome the limitation, we propose a novel method of synthesizing a facial image with micro-level facial elements. A deep learning-based high-resolution image synthesis model is employed after generating a label image from the face RGB image through skin micro-element segmentation and face attribute segmentation. In the process of generating a label image for synthesizing skin micro-elements, we propose a technique for controlling skin micro-elements, enabling the generation of various label images from a single face label image. Throughout the proposed method, the areas of skin micro-elements can be edited and different skin types can be simulated. The experimental results show that the generated face is significantly improved by applying the proposed method. Moreover, various faces can be synthesized by changing the types and stages of skin micro-elements.