Learning monocular face reconstruction from in the wild images using rotation cycle consistency

Abstract

With the popularity of the digital human body, monocular three-dimensional (3D) face reconstruction is widely used in fields such as animation and face recognition. Although current methods trained using single-view image sets perform well in monocular 3D face reconstruction tasks, they tend to rely on the constraints of the a priori model or the appearance conditions of the input images, fundamentally because of the inability to propose an effective method to reduce the effects of two-dimensional (2D) ambiguity. To solve this problem, we developed an unsupervised training framework for monocular face 3D reconstruction using rotational cycle consistency. Specifically, to learn more accurate facial information, we first used an autoencoder to factor the input images and applied these factors to generate normalized frontal views. We then proceeded through a differentiable renderer to use rotational consistency to continuously perceive refinement. Our method provided implicit multi-view consistency constraints on the pose and depth information estimation of the input face, and the performance was accurate and robust in the presence of large variations in expression and pose. In the benchmark tests, our method performed more stably and realistically than other methods that used 3D face reconstruction in monocular 2D images.