Domain-adaptive point cloud semantic segmentation via knowledge-augmented deep learning

Abstract

Owing to limitations in both the quality and quantity of training data, supervised deep learning methods for point cloud semantic segmentation often exhibit poor generalization and hallucinations, particularly when unstructured scenes are processed in autonomous driving applications. These challenges motivate us to develop a domain-adaptive point cloud semantic segmentation network based on knowledge-augmented deep learning, which can be applied to off-road scenes after being trained on urban datasets. Specifically, our method adopts the strategy of combining implicit and explicit knowledge augmentation. Moreover, network modulation based on attribution analysis is employed to integrate domain knowledge into the deep learning model, thereby mitigating the model’s susceptibility to spurious correlations. Additionally, we propose new network modulation performance metrics to evaluate the efficiency and benefit of modulation correction. For experimental validation, we use datasets with significant disparities between the urban and off-road environments in the training and testing phases, respectively. Furthermore, we release our self-collected off-road dataset PCSS-Gobi-3D, which is the first point cloud dataset of the Gobi Desert scene. Compared with other domain-adaptive methods, our method demonstrates promising cross-domain 3D semantic segmentation performance with excellent results in terms of mean Intersection over Union (mIoU) and overall accuracy.