Optimization of multi-prior 3D human reconstruction methods based on single-view images

Abstract

The simultaneous recovery of a human body’s 3D shape and surface color from a single image is a challenging task with numerous applications. To improve the accuracy of single-view 3D human reconstruction, we propose a comprehensive optimization method aimed at enhancing the quality of human shape and surface texture recovery. First, to address the issue of insufficient texture details in the invisible regions, we fuse rearview normal features with the SMPL-X human model features and front normal features as additional prior information. This fusion enhances texture reconstruction details in the invisible areas by local feature enhancement during normal map generation. Second, to tackle the problem of missing hands, we introduce the MediaPipe Hands keypoints detection algorithm. This algorithm optimizes the hand replacement process by accurately determining the visibility of the hands, ensuring high-quality replacement for the hands of the 3D human model. Finally, during the stage of 3D human model refinement, we implement an outlier removal algorithm. This algorithm effectively eliminates fragments from the edges of the 3D human model and optimizes the frontal texture by employing color texture mapping, which projects image pixel color information onto the surface of the 3D human model. Experimental results demonstrate that our proposed method outperforms existing techniques in terms of 3D human model shape recovery and surface texture fidelity, providing a novel solution for advancing 3D human reconstruction technology.