AfaMamba: Adaptive Feature Aggregation With Visual State Space Model for Remote Sensing Images Semantic Segmentation

Abstract

Remote sensing images semantic segmentation is typically challenging due to the complexity of land cover information. Existing convolutional neural network (CNN)-based models lack the capability to model long-range dependencies, while Transformer-based models are constrained by quadratic computational complexity. Recently, an advanced visual state space model known as the Mamba architecture has been introduced, which ensures linear computational complexity while effectively extracting global contextual information. However, the Mamba architecture lacks the ability to model fine-grained local information, thereby failing to fully leverage both global and local contextual information. To address these issues, we propose a novel network called adaptive feature aggregation with Mamba (AfaMamba). It employs a lightweight ResNet18 as the encoder, and during the decoding phase, it first utilizes a multiscale feature adaptive aggregation module to ensure that the output features from each stage of the encoder contain rich multiscale semantic information. Subsequently, the global-local Mamba structure combines the attention-optimized multiscale convolutional branches with the global branch of Mamba to facilitate effective interaction between global and local features. In addition, a lightweight CNN stem is introduced to extract shallow image features, enhancing the model's ability to capture spatial detail information. Extensive experiments conducted on two widely used remote sensing datasets, ISPRS Potsdam and LoveDA, demonstrate that AfaMamba achieves a superior balance between accuracy and efficiency compared to state-of-the-art models.