Multi-view stereo with cross-scale feature fusion strategy and hybrid depth estimation

Abstract

In multi-view stereo (MVS) 3D reconstruction, existing methods often face challenges such as insufficient feature representation in weakly textured areas, assumptions of equal view contributions, and limited depth estimation accuracy, leading to incomplete reconstruction results. To address these issues, we propose a multi-view stereo method integrating a cross-scale feature fusion strategy and hybrid depth estimation (CH-MVSNet), aimed at improving the precision and completeness of MVS reconstruction. Our approach introduces a multi-scale feature enhancement module (MFEM), which combines channel attention mechanisms with multi-scale feature fusion to enhance features from source and reference images, improving intra-image contextual information and inter-image feature relationships. We then propose a weighted view cost volume module (WVCM), which calculates weighted view correlations to construct a more precise cost volume, further improving reconstruction accuracy. Finally, we incorporate an RGB-guided hybrid depth estimation module (RHDE), which combines classification and regression methods for depth estimation, utilizing RGB information from reference images to optimize the depth map precision. Through rigorous testing on the DTU dataset and Tanks and Temples benchmark, our method demonstrates significant improvements in reconstruction accuracy and completeness.