Consistent Long Sequences Deep Faces

Abstract

Face swapping in videos usually has strong entertainment applications. Deep Fakes (in Faces) are a recent topic in deep learning where the main idea is to substitute the face of a person in a video with the face of another person. But one of the drawbacks of the method is that between two successive frames there are inconsistencies between the faces, such as changing face color, flickering or eyebrows that change. In this paper, we propose a convolutional neural network for swapping faces based on two autoencoders which share the same encoder. In this network, the encoder can distinguish and extract important features of faces, including facial expressions and poses; the decoders will then reconstruct faces according to these features. First, we will generate datasets of faces respectively for person A and person B. Secondly, the local information of two faces is sent to the network to get the model; after the training process, we can use the model to reconstruct the corresponding face of person B when the input is one face of person A. Afterwards, we build a binary mask to select the face area and transfer color from the source face to the target face. Finally, we only need to use a seamless clone to merge the new faces back into the source frames to create a fake video. The experimental results show that the quality of the fake videos is improved significantly.