Breaking the limitations of scenes and sensors variability: A novel unsupervised domain adaptive instance segmentation framework for agricultural field extraction

Abstract

Extraction of agricultural field parcels is of great importance for agricultural condition monitoring, farm management, and food security. Several methods have been developed to map the distribution of agricultural field parcels, among which deep learning-based supervised learning is increasingly employed. Nevertheless, advanced deep learning models face two major limitations: limited ability to generalize across different spatial，temporal and sensor contexts with varying scene and object characteristics, and high requirement for annotated datasets to support training and validation. To address this challenge, we introduce a novel unsupervised domain adaptation (UDA) framework (UDA-Field Teacher, UDA-FT) for agricultural field parcel instance segmentation, which is designed to transfer knowledge from labeled source domains to unlabeled target domains. UDA-FT is based on the Mask R-CNN framework and incorporates a target-oriented teacher model and a cross-domain student model. This cross-domain student model embeds an image adaptation module and an instance adaptation module, employing adversarial learning strategies to mitigate cross-domain distribution differences. Additionally, we propose a consistency mutual learning module based on soft pseudo-label technology, overcoming the limitations of traditional hard pseudo-labeling in confidence threshold selection and improving model robustness in the target domain. Furthermore, to address the difficulty in generating independent instance labels for densely packed agricultural field parcels and capturing spatial contextual relationships during soft pseudo-label generation, we propose two data augmentation methods, namely CutMatch (CM) and LeakyMask (LM). We adopted the proposed framework on cross-scene and cross-sensor datasets to evaluate its effectiveness and robustness under different scenes. Quantification and visualization results demonstrate our UDA-FT outperforms existing domain adaptation methods for cross-scene and cross-sensor agricultural field parcels across all metrics. Ablation studies highlight the substantial impact of strong data augmentation on model performance, emphasizing the importance of learning from out-of-distribution data. As an innovative application of unsupervised domain adaptation in agricultural field parcel instance segmentation, this research provides a novel method for domain shift in agricultural remote sensing imagery, enabling more accurate field instance segmentation with significant implications for global agriculture.