OrthoMatch-Net: Unsupervised registration of orthodontic dental point clouds via hierarchical attention feature modeling and bidirectional matching mechanism

Accurate 3D dental point cloud registration is a crucial task for monitoring therapeutic progress and evaluating treatment outcomes. However, existing methods struggle to align dental structures owing to their intricate, highly similar shapes, as well as noise and pose variations in clinical environments, and are hindered by inadequate feature extraction and insufficient modeling of feature interactions. To tackle these challenges, we propose OrthoMatch-Net, an innovative unsupervised framework for dental point cloud registration, whose core contributions lie in two novel designs: (1) hierarchical attention feature modeling and (2) bidirectional matching mechanism, aimed at achieving robust alignment of pre- and post-treatment dental point clouds. The proposed hierarchical attention feature modeling employs transformation-invariant guided cross-attention to enhance local feature aggregation. It further captures global structural relationships through the window transformer. Moreover, a feedback interaction mechanism is introduced to enable feature fusion across hierarchical levels, thereby improving discriminative representation robustness for dental registration. Simultaneously, the bidirectional matching mechanism reinforces geometric consistency by learning key point correspondences in both directions (source-to-target and target-to-source). It leverages local structural consistency to weight and filter the matched pairs, effectively enhancing the symmetry and stability of the registration process. Extensive experiments on clinical dental datasets demonstrate that OrthoMatch-Net outperforms state-of-the-art methods with sub-millimeter accuracy across multiple metrics. It also exhibits strong robustness under noise perturbations, offering a practical and reliable solution for improving orthodontic treatment precision and supporting clinical decision-making. To facilitate further study, our source code and the pretrained models will be released at https://github.com/shanshanhuang2023/OrthoMatch-Net.