3D Face Reconstruction via Feature Point Depth Estimation and Shape Deformation

Abstract

Since a human face can be represented by a few feature points (FPs) with less redundant information, and calculated by a linear combination of a small number of prototypical faces, we propose a two-step 3D face reconstruction approach including FP depth estimation and shape deformation. The proposed approach can reconstruct a realistic 3D face from a 2D frontal face image. In the first step, a coupled dictionary learning method based on sparse representation is employed to explore the underlying mappings between 2D and 3D training FPs, and then the depth of the FPs is estimated. In the second step, a novel shape deformation method is proposed to reconstruct the 3D face by combining a small number of most relevant deformed faces by the estimated FPs. The proposed approach can explore the distributions of 2D and 3D faces and the underlying mappings between them well, because human faces are represented by low-dimensional FPs, and their distributions are described by sparse representations. Moreover, it is much more flexible since we can make any change in any step. Extensive experiments are conducted on BJUT_3D database, and the results validate the effectiveness of the proposed approach.