Multi-modal vision-driven point cloud registration for efficient fusion of multi-source models in regional building clusters

Abstract

Integrating the OPM (Oblique Photogrammetry Model) and BIM (Building Information Model) is a critical challenge in advancing smart city due to difficulties in multi-scale heterogeneous data fusion. This paper presents a method to improve the efficiency and accuracy of automatic multi-source model fusion in regional building clusters. The proposed framework integrates YOLOv10 and SAM to detect and segment building contours from multi-modal images. A ray-tracing method is then applied to unitize buildings within the OPM, enabling accurate localization. To ensure scale consistency, a ring-scanning strategy performs resolution-based sampling of exterior surfaces from both unitized OPM and BIM. For fusion, computer vision algorithms conduct point cloud denoising and coarse registration, which is further refined using the Iterative Closest Point (ICP) algorithm. This method enables seamless fusion of multi-source models into a unified, closed-loop digital twin base, establishing a robust foundation for high-precision data integration and visualization in smart city applications.