Context-aware adaptive network for UDA semantic segmentation

Abstract

Unsupervised Domain Adaptation (UDA) plays a pivotal role in enhancing the segmentation performance of models in the target domain by mitigating the domain shift between the source and target domains. However, Existing UDA image mix methods often overlook the contextual association between classes, limiting the segmentation capability of the model. To address this issue, we propose the context-aware adaptive network that enhances the model’s perception of contextual association information and maintains the contextual associations between different classes in mixed images, thereby improving the adaptability of the model. Firstly, we design a image mix strategy based on dynamic class correlation called DCCMix that constructs class correlation meta groups to preserve the contextual associations between different classes. Simultaneously, DCCMix dynamically adjusts the class proportion of the source domain within the mixed domain to gradually align with the distribution of the target domain, thereby improving training effectiveness. Secondly, the feature-wise fusion module and contextual feature-aware module are designed to better perceive contextual information of images and alleviate the issue of information loss during the feature extraction. Finally, we propose an adaptive class-edge weight to strengthen the segmentation ability of edge pixels in the model. Experimental results demonstrate that our proposed method achieves the mloU of 63.2% and 69.8% on two UDA benchmark tasks: SYNTHIA \(\rightarrow\) Cityscapes and GTA \(\rightarrow\) Cityscapes respectively. The code is available at https://github.com/yuheyuan/CAAN.