S2Reg: Structure-semantics collaborative point cloud registration

Abstract

Point cloud registration is one of the essential tasks in 3D vision. However, most existing methods mainly locate the point correspondences based on geometric information or adopt semantic information to filter out incorrect correspondences. They overlook the underlying correlation between semantics and structure. In this paper, we propose a structure-semantics collaborative point cloud registration method. Firstly, we propose a Superpoint Semantic Feature Representation module (SSFR), which incorporates multiple semantics of neighboring points to characterize the semantics of superpoints. Then, through a Structural and Semantic Feature correLation with Attention Guidance module (S${}^2$FLAG), we capture the global correlation of semantics and structure within a point cloud, as well as the consistency of semantics and structure between point clouds. Moreover, an image semantic segmentation foundation model is employed to acquire semantics when images of the point clouds are available. Extensive experiments demonstrate that our method achieves superior performance, especially in low-overlap scenarios. Our code and models are available at https://github.com/GAOXINYU203/s2reg.