Cross-Domain Multi-Prototypes with Contradictory Structure Learning for Semi-Supervised Domain Adaptation Segmentation of Remote Sensing Images

Abstract

Recently, unsupervised domain adaptation (UDA) segmentation of remote sensing images (RSIs) has attracted a lot of attention. However, the performance of such methods still lags far behind that of their supervised counterparts. To this end, this paper focuses on a more practical yet under-investigated problem, semi-supervised domain adaptation (SSDA) segmentation of RSIs, to effectively improve the segmentation results of targeted RSIs with a few labeled samples. First, differently from the existing single-prototype mode, a novel cross-domain multi-prototype constraint is proposed, to deal with large inter-domain discrepancies and intra-domain variations. Specifically, each class is represented as a set of prototypes, so that multiple sets of prototypes corresponding to different classes can better model complex inter-class differences, while different prototypes within the same class can better describe the rich intra-class relations. Meanwhile, the multi-prototypes are calculated and updated jointly using source and target samples, which can effectively promote the utilization and fusion of the feature information in different domains. Second, a contradictory structure learning mechanism is designed to further improve the domain alignment, with an enveloping form. Third, self-supervised learning is adopted, to increase the number of target samples involved in prototype updating and domain adaptation training. Extensive experiments verified the effectiveness of the proposed method for two aspects: (1) Compared with the existing SSDA methods, the proposed method could effectively improve the segmentation performance by at least 7.38%, 4.80%, and 2.33% on the Vaihingen, Potsdam, and Urban datasets, respectively; (2) with only five labeled target samples available, the proposed method could significantly narrow the gap with its supervised counterparts, which was reduced to at least 4.04%, 6.04%, and 2.41% for the three RSIs.