Semantic Segmentation of High-Resolution Remote Sensing Imagery via an End-to-End Graph Attention Network With Superpixel Embedding

Abstract

Semantic segmentation of high-resolution remote sensing images is crucial in ecological evaluation, natural resource surveys, etc. Compared with CNN-based and transformer-based methods, graph neural networks (GNNs) have drawn increasing attention because they can flexibly model topologies of arbitrary irregular objects on graphs. Researchers typically use superpixels as graph nodes to reduce image noise and computational complexity. However, most superpixel-based GNN methods view superpixel segmentation as a data preprocessing step. This results in fixed graphs input to GNNs and overlooks the effects of undersegmentation. In addition, these methods often employ one graph construction approach, which makes them susceptible to interclass similarity (ICS) or intraclass variability (ICV), leading to segmentation inaccuracies. To address these issues, we propose an end-to-end graph attention network with superpixel embedding (SEGAT) to achieve semantic segmentation with well-delineated boundaries. We first use a learnable neural network, the superpixel generation module (SGM), to generate superpixels, which is cotrained with subsequent graph segmentation module (GSM) to refine boundaries continuously. Dynamically fine superpixels produce dynamically optimized graphs and mitigate undersegmentation errors. To reduce the interference of ICS and ICV, we then use the GSM to construct local and global graphs based on superpixel spatial positions and feature similarity, respectively, and update superpixel features and graph structure. Finally, updated superpixel features are classified for superpixel-wise classification, which is then mapped back to pixel features through the pixel-superpixel association map. Extensive experiments on three datasets, Vaihingen, Potsdam, and UAVid, demonstrate that SEGAT can outperform state-of-the-art methods.