Efficient semantic segmentation of remote sensing images through dynamic feature enhancement and multimodal alignment fusion

Multimodal fusion has shown promising applications in integrating information from different modalities. However, existing multimodal fusion approaches in remote sensing face two main challenges: First, multimodal fusion models relying on Convolutional Neural Networks (CNNs) or Visual Transformers (ViTs) have limitations in terms of remote modeling capabilities and computational complexity, while state-space model (SSM)-based fusion models are prone to feature redundancy due to the use of multiple scanning paths, and similarly suffer from high computational complexity. Second, existing methods do not fully address inter-modal heterogeneity, leading to poor multimodal data fusion. To address these issues, we propose an efficient multimodal fusion network, AFMamba, based on the state-space model (SSM) for semantic segmentation of remote sensing images. Specifically, we design the Efficient Dynamic Visual State Space (EDVSS) module, which enhances the efficiency of the standard Mamba model by dynamically improving local features and reducing channel redundancy. Furthermore, we introduce the Cross Attention Alignment Fusion (CAAFM) module, which combines cross-image attention fusion and channel interaction alignment to effectively improve the accuracy and efficiency of cross-modal feature fusion and mitigate feature inconsistency. Experimental results demonstrate that in multimodal hyperspectral image semantic segmentation, the proposed model reduces computational complexity, measured in GFLOPs, by at least 61 % while maintaining a low parameter count, achieving optimal overall accuracy (OA) of around 92 %, and effectively balancing performance and computational efficiency.