3D surface reconstruction with enhanced high-frequency details

Abstract

Neural implicit 3D reconstruction can reproduce shapes without the need for 3D supervision, making it a significant advancement in computer vision and graphics. This technique leverages volume rendering methods and neural implicit representations to learn and reconstruct 3D scenes directly from 2D images, enabling the generation of complex geometries and detailed structures with minimal data. The field has gained significant traction in recent years, due to advancements in deep learning, 3D vision, and rendering techniques that allow for more efficient and realistic reconstructions. Current neural surface reconstruction methods tend to randomly sample the entire image, making it difficult to learn high-frequency details on the surface, and thus the reconstruction results tend to be too smooth. We designed a method, termed FreNeuS (Frequency-guided Neural Surface Reconstruction), which leverages high-frequency information to address the problem of insufficient surface detail. Specifically, FreNeuS uses pixel gradient changes to easily acquire high-frequency regions in an image and uses the obtained high-frequency information to guide surface detail reconstruction. High-frequency information is first used to guide the dynamic sampling of rays, applying different sampling strategies according to variations in high-frequency regions. To further enhance the focus on surface details, we have designed a high-frequency weighting method that constrains the representation of high-frequency details during the reconstruction process. Compared to the baseline method, Neus, our approach reduces the reconstruction error by 13% on the DTU dataset. Additionally, on the NeRF-synthetic dataset, our method demonstrates a significant advantage in visualization, producing clearer texture details. In addition, our method is more applicable and can be generalized to any reconstruction method based on NeuS.