Semantic Co-Occurrence and Relationship Modeling for Remote Sensing Image Segmentation

Abstract

Semantic segmentation is an important but challenging task in pixel-level remote sensing (RS) data analysis. Accurate segmentation is essential for applications such as land use classification, infrastructure monitoring, and environmental conservation. However, RS semantic segmentation is hindered by issues like class imbalance, occlusion, blurring, and small target sizes. Existing models are lacking the capability to capture and utilize contextual and semantic relationships between different object classes. To overcome these challenges, we propose an enhanced semantic segmentation framework that integrates domain-specific knowledge through our Semantic Co-occurrence and Relationship Module (SCRM). The SCRM comprises two key components: a Probabilistic Co-occurrence Knowledge Module that incorporates statistical class correlations into the training process, and an Inter-Class Feature Relationship Module that models feature-level interactions between classes. By embedding SCRM into both classic and state-of-the-art segmentation models, our method leverages contextual relationships to improve segmentation performance. We evaluate our method on four RS datasets: two RGB-T (KUST4K and MFNet) and two RGB (aeroscapes and DLRSD). Experimental results demonstrate that our enhanced models achieve significant improvements in mAcc and mIoU across all datasets and baseline methods.