Accurate matching between BIM-rendered and real-world images

Abstract

—As the digital representation of the built environ- ment, BIM has been used to assist robot localization. Real-world images captured by the robot camera can be compared with BIM-rendered images to estimate the pose. However, there is a perception gap between the BIM environment and reality; image styles are typically too different to be matched. Hence, this study investigates an advanced image feature detection technique, D2-Net, to identify key points and descriptors on BIM-rendered and real-world images. These key features are further matched via K Nearest Neighbor Search and RANSAC. The ability to bridge the perception gap can be evaluated by the image matching performance, which is the Euclidean distance between the projected key points and the number of inliers. SIFT, as the traditional feature detection technique, was compared in this study. Results show that the average projection error of D2-Net is only 16.55 pixels, while the error of SIFT is 187.46 pixels. It demonstrates that the advanced D2-Net can be utilized to detect representative features on BIM-rendered and real-world images. The matched image pairs can be further utilized to estimate the robot pose in BIM. Overall, it aims to enhance the BIM-assisted localization and improve the robot’s reliability as a decision-making tool on-site.