Research on Cartoon Face Generation Based on CycleGAN Assisted with Facial Landmarks

Abstract

Turn real faces into cartoon faces is a topic of style transfer, and style transfer is a hot topic in the application of generative adversarial networks in image. CycleGAN is one of generative adversarial networks. It has obvious universal applicability, and has a good transformation effect on various types of style transfer. But to the facial style transfer, it only focuses on the transformation of the whole face, and it is not ideal for the transformation of the details of the facial features. How can this situation be improved? In this paper, we use facial landmarks to assist the transformation of facial features. In the beginning, we use stacked hourglass networks to detection and capture landmarks of real faces. And then, use them to assist cartoon faces generation. In view of the fact that the hourglass network has its own advantages in feature extraction, we use it to replace the generator structure of the original CycleGAN for transformation. And in order to avoid the Checkerboard Artifacts and ensure the quality of image generation, we use bilinear interpolation in the upsampling part of the generator to replace the deconvolution of the original generator and the nearest interpolation of the hourglass network. Experiments show that these practices have good results in optimizing conversion performance and improving image quality.