Boosting robust multimodal point cloud registration via tightly-coupled framework of visual and geometric features

In large-scale scene reconstruction using multimodal image-point cloud sensors, point cloud registration techniques face significant challenges in scenarios characterized by weak texture, sparse geometry, limited overlap, and strong noise interference. In this paper, we present a tightly-coupled visual-geometric feature fusion framework to overcome these limitations, thereby enhancing the robustness and accuracy of registration in challenging scenarios. A density-guided feature sampling method based on radiation field is proposed to address the non-uniform sampling issue inherent in single-sensor vulnerable scenarios. Then, an adaptive weighting strategy for tight visual-geometric feature coupling is introduced to improve performance in scenes with weak texture and geometry. Finally, a visual and geometric fusion correspondence sampling method based on maximal cliques is proposed to enhance matching performance in scenarios with limited overlap and strong noise interference. Notably, the proposed framework can serve as a plug-in to improve the performance of visual and geometric fusion registration algorithms, and the feature detectors used within the framework can be replaced by any feature descriptor-based feature detection method. The proposed method enables multimodal image-point cloud sensors to achieve robust registration in various extreme scenarios, overcoming the potential registration failures inherent in conventional approaches. Experimental results demonstrated that under challenging scenarios, the proposed method improved the inlier ratios of features by 8.37 %, reduced rotational errors by 78.85 %, and decreased translational errors by 85.19 % compared to other methods. This advancement establishes a high-precision point cloud registration framework for large-scale scene reconstruction, exhibiting both robustness and metrological superiority.