Weather-aware autopilot: Domain generalization for point cloud semantic segmentation in diverse weather scenarios

Abstract

3D point cloud semantic segmentation, a pivotal task in fields such as autonomous driving and urban planning, confronts the challenge of performance degradation under adverse weather conditions. Current methodologies primarily focus on optimal weather scenarios, leaving a significant gap in handling various environmental adversities like fog, rain, and snow. To bridge this gap, we propose a comprehensive deep learning framework featuring unique components — an Adaptive Feature Normalization Module (AFNM) for effective normalization and calibration of features, a Dual-Attention Fusion Module (DAFM) for integrating cross-domain features, and a Proxy Label Generation Module (PLGM) for generating reliable proxy labels within the domain. Utilizing the SemanticKITTI and SynLiDAR datasets as source domains and the SemanticSTF dataset as the target domain, our model has been rigorously evaluated under varying weather conditions. When trained on the SemanticKITTI dataset as the source domain with the SemanticSTF dataset as the target, our approach surpasses the current state-of-the-art models by a margin of 6.2% in terms of overall mean Intersection over Union (mIoU) scores. Similarly, with the SynLiDAR dataset as the source and SemanticSTF as the target, our performance exceeds the best existing models by 3.4% in mIoU. These results substantiate the efficacy of our model in advancing the field of 3D semantic segmentation under diverse weather conditions, showcasing its notable robustness and superiority. The code is available at https://github.com/J2DU/WADG-PointSeg.