Strategic sampling for training a semantic segmentation model in operational mapping: Case studies on cropland parcel extraction

Abstract

Semantic segmentation of remotely sensed images has become increasingly popular for a wide range of natural resource and urban application, yielding promising results. To an operational semantic segmentation mapping project, having more samples generally enables the model to better extract target features, achieving higher accuracies. However, annotating remote sensing image samples for model training is a time-consuming and labor-intensive process. Strategic sampling aims to minimize the efforts in collecting new training samples for a mapping project, which has been not well studied yet for semantic segmentation. To approach this topic, we employed a hybrid way for combining meta-analysis and case studies to investigate the best practices for strategic sampling. Three factors relating to strategic sampling will be investigated: sample size, distribution and transferring methods. We first reviewed 334 recently published papers that adopted semantic segmentation for operational mapping projects to summarize the current status of training sample design from various mapping scenarios. Subsequently, we constructed a large dataset of over 12,000 high-quality annotated image patches for cropland parcel mapping across five study sites, and evaluated various sampling strategies using a baseline segmentation model. We also proposed a novel balanced sampling method, which leveraged patch-based entropy and edge complexity to classify sample diversity. Our findings revealed that (1) both meta-analysis and the case studies suggested that ∼4 % of the total mapping patches were the optimal training sample size under random sampling, i.e., the minimum size to reach accuracy saturation; (2) compared to random sampling, the newly proposed balanced sampling was superior due to its decreasing the required sample size from ∼4 % to 2.5 % of the total patches in mapped areas; (3) sample transfer and model transfer present identical performance for relaxing the average local sample demand from 2.5 % to 0.5 % of total patches, with sample transfer being slightly more accurate than model transfer (Global Total-Classification errors: 0.298 vs 0.308). This study offers a heuristic framework for applying strategic sampling in semantic segmentation, providing valuable practical guidance for implementing deep learning in an operational scenario.