Test Time Adaptation with An Explicit Geometric Bridge for 3D Point Cloud Segmentation.

Abstract

Test Time Adaptation (TTA) is a flexible unsupervised domain generalization method that allows models to adapt to the target domain distribution without requiring access to the source domain data and target domain labels. However, when applied to 3D point cloud scene segmentation, the significant domain gaps introduced by complex scenes and the common issue of class imbalance in segmentation tasks can lead to training instabilities in previous TTA methods, resulting in a degradation of the performance of the source model. To address this issue, we consider that explicit geometric features in the target domain can serve as an intermediate bridge that roughly approximates the target domain to provide strong geometric prior of the target domain data distribution, thereby reducing the domain gap and enhancing domain adaptation performance. Therefore, we propose TTA-EGB that uses Explicit Geometric feature as the Bridge to stabilize the Test Time Adaption process. Specifically, in TTA-EGB, we first introduce a non-parametric geometry model to extract explicit geometric feature and use knowledge distillation to guide the source model to stably adapt to the target domain. Furthermore, previous TTA methods often employ a class-agnostic sampling mechanism to select easy samples for optimization, which causes the model to be overly confident in the major classes and neglect minority classes. Therefore, we propose a category-balanced sampling mechanism that dynamically defines sampling thresholds based on class frequencies, which results in a similar proportion of classes in both easy and hard samples, thereby improving the issue of class imbalance. We conduct the main experiments on the more challenge sim-to-real benchmark about synthetic dataset 3DFRONT and the real-world datasets ScanNet and S3DIS for 3D segmentation task. Results show that our method can efficiently improve the mIOU by over 3% on 3DFRONT→ ScanNet and 7% on 3DFRONT→ S3DIS.