Face reconstruction with detailed skin features via three selfie images

Abstract

Accurate 3D reconstruction of facial skin features, such as acne, pigmentation, and wrinkles, is essential for digital facial analysis, virtual aesthetics, and dermatological diagnostics. However, achieving high-fidelity skin detail reconstruction from limited, in-the-wild inputs like selfie images remains a largely underexplored challenge. The Hierarchical Representation Network (HRN) excels in reconstructing facial geometry from limited images but faces challenges in skin detail fidelity and multi-view matching. In this work, we present a lightweight and deployable system that reconstructs detailed 3D face models from only three guided portrait images. We address these limitations by enhancing HRN’s output resolution, improving skin detail precision, and introducing a novel multi-view texture map fusion framework with illumination normalization and linear blending, enhancing texture clarity. To correct eye direction inconsistencies, we integrate a segmentation network to refine eye regions. We further develop a mobile-based prototype application that guides users through video-based face capture and enables real-time model generation. The system has been successfully applied in real-world settings. Our dataset, featuring annotated portraits of fair-skinned Asian females, with visible skin conditions, serves as a benchmark for evaluation. This is the first benchmark focusing on skin-level 3D reconstruction from selfie-level inputs. We validated our method through ablation, comparison, and perception studies, all of which demonstrated clear improvements in texture fidelity and fine detail. These results indicate the method’s practical value for 3D facial skin reconstruction.