Coarse-to-fine online latent representations matching for one-stage domain adaptive semantic segmentation

Abstract

Domain adaptive semantic segmentation is meaningful since collecting numerous labeled samples in different domains is expensive and time-consuming. Recent domain adaptation methods yield not so efficient performance compared with supervised learning. With the hypothesis that semantic feature can be shared across domains, this paper proposes a coarse-to-fine online matching architecture (COM) for one-stage domain adaptation. We consider subsequent learning stages progressively refining the task in the latent feature space, i.e. the finer set at each component is hierarchically derived from the coarser set of the previous components, including cross-domain global prototypes, categories and instances matching and anchor-points contrastive learning, which further achieve self-supervised learning with region-level pseudo label generated only in a single training step. Beforehand, feature refinement are performed to realize edge perception and inter-feature augmentation. Then, coarse-to-fine network fuses global and local consistency matching via specific distribution alignment between the source and target domain. Finally, the adversarial structure controls the uncertainty of generator prediction through the maximization of classification results and minimization of two classifiers discrepancy. This proposed method is evaluated in two unsupervised domain adaptation tasks, i.e. GTA5 $\to$ Cityscapes and SYNTHIA $\to$ Cityscapes. Extensive experiments verify the effectiveness of our proposed COM model and demonstrate its superiority over several state-of-the-art approaches.