Towards Accurate 3D Human Reconstruction: Segmentation-Based Supervision With Uncertainty Estimation

Abstract

Human body reconstruction leveraging image information has become a critical task in the signal processing community. Due to the scarcity of high-quality 3D labels, existing methods often neglect the impact of body shape on the realism of the reconstruction. We argue that parameterized human models (such as SMPL) can control the reconstructed body shape through parameters, a feature that is underutilized in most reconstruction systems. Therefore, we design an end-to-end 3D parameterized human reconstruction system capable of real-time reconstruction of realistically shaped human models. To meet system requirements, we propose the Segmentation-based Supervision with Uncertainty Estimation (SSUE) framework, which innovatively employs body part segmentation as supervisory information and mitigates the adverse effects of segmentation noise through uncertainty estimation. Experimental results demonstrate improvements of 3.2% over the SOTA methods in body shape reconstruction accuracy and enhancements in the precision of limb extremities with our SSUE framework.