Learning hierarchical uncertainty from hybrid representations for neural active reconstruction

Abstract

Active reconstruction is a key area for the robotics and computer vision communities, enabling autonomous agents to dynamically reconstruct scenes or objects from multiple viewpoints for navigation and manipulation tasks. Although existing methods have achieved promising results in 3D reconstruction, the hierarchical uncertainty-aware active reconstruction based on hybrid implicit representations remains underexplored, particularly in balancing accuracy, efficiency, and adaptability. To address this gap, we propose a neural active reconstruction system that combines hybrid neural representations with uncertainty. Specifically, we explore a novel scheme that integrates occupancy, signed distance function, and neural radiance fields for high-fidelity 3D reconstruction. Additionally, we utilize hierarchical uncertainty associated with different representations to select the next best viewpoint for trajectory planning and optimization. Our system has been extensively evaluated on benchmark datasets including Replica and MP3D, demonstrating qualitatively and quantitatively improved reconstruction quality and view planning efficiency compared to baseline approaches.