Test-time adaptation for geospatial point cloud semantic segmentation with distinct domain shifts

Abstract

Domain adaptation (DA) techniques aim to close the gap between source and target domains, enabling deep learning models to generalize across different data shift paradigms for point cloud semantic segmentation (PCSS). Among emerging DA schemes, test-time adaptation (TTA) facilitates direct adaptation of a pre-trained model to unlabeled data during the inference stage without access to source domain data and need for additional training process, which mitigates data privacy concerns and removes the requirement for substantial computational power. To fill the gap of leveraging TTA for geospatial PCSS, we introduce three typical domain shift paradigms in handling geospatial point clouds and construct three practical adaptation benchmarks, including photogrammetric point clouds to airborne LiDAR, airborne LiDAR to mobile LiDAR, and synthetic to mobile LiDAR. Then, a TTA method is proposed by exploiting the domain-specific knowledge embedded within the batch normalization (BN) layers. Given the pre-trained model, BN statistical information is progressively updated by fusing the statistics of each testing batch. Furthermore, we develop a self-supervised module to optimize the learnable BN affine parameters. Information maximization is used to generate confident and category-specific predictions, and reliability constrained pseudo-labeling is further incorporated to create supervisory signals. Extensive experimental analysis demonstrates that our proposed method significantly improves classification accuracy compared to directly applying the inference by up to 20% in terms of mIoU, which not only outperforms other popular counterparts but also maintains a high efficiency while avoiding retraining. In an adaptation of photogrammetric (SensatUrban) to airborne (Hessigheim 3D), our method achieves a mIoU of 59.46% and an OA of 85. 97%.