FineCrop: Mapping fine-grained crops using class-aware feature decoupling and parcel-aware class rebalancing with Sentinel-2 time series

Fine-grained crop mapping refers to the precise differentiation of all crop types within an area, encompassing major classes (e.g., staple crops, to cash crops, to garden fruits, etc.) and their subclasses (e.g., wheat, barley, maize of staple crops). Fine-grained crop mapping is crucial for precise agriculture management. However, compared with staple crop mapping, fine-grained crop mapping faces more challenges: (1) the extremely similar phenological characteristics between crop subcategories, which could lead to the difficulty in extracting discriminative representation; (2) the imbalanced class distribution, which could lead to the bias of the model toward the head class, finally causing severe misclassification. In this paper, we proposed a novel framework for fine-grained crop mapping, termed FineCrop, by using class-aware feature decoupling (CFD) branch and parcel-aware class rebalancing (PCR) branch. Specifically, CFD was inspired by the “divide and conquer” theory and designed to learn the detailed and independent features of each crop type and to solve the phenological similarity. PCR was inspired by the data aggregation and designed to use a class-aware factor at parcel unit to solve the bias of classifier caused by the imbalanced data distribution. To evaluate FineCrop, we have built a fine-grained crops mapping dataset, termed FineCropSet by matching Sentinel-2 Level-2A product that has undergone radiometric and geometric correction with labels extracted from the EuroCrops. FineCropSet contains 138 crop types covering the North Rhine-Westphalia, the south of Slovakia, and the Netherlands of different years. The results showed that FineCrop can improve the overall accuracy of popular deep learning models for temporal satellite imagery by 5.83 %, 1.42 %, 0.89 % for three study areas respectively (p-value less than 0.05) verified by paired t-test, confirming the substantial improvement of FineCrop for fine-grained crop mapping. The ablation experiment results revealed that FineCrop could reduce the class imbalance by 5 and 18 times and extract the detailed features in parcel from edge to center. We believe the proposed method is promising for large-scale crop mapping particularly when dealing with crops with similar phenological characteristics and imbalanced distribution, leading to more accurate crop resources inventory. The source code and data are available: https://github.com/LL0912/FineCrop.