DualMamba: A Lightweight Spectral–Spatial Mamba-Convolution Network for Hyperspectral Image Classification

Abstract

The effectiveness and efficiency of modeling complex spectral–spatial relations are crucial for hyperspectral image (HSI) classification. Most existing methods based on convolution neural networks (CNNs) and transformers still suffer from heavy computational burdens and have room for improvement in capturing the global–local spectral–spatial feature representation. To this end, we propose a novel lightweight parallel design called a lightweight dual-stream Mamba-convolution network (DualMamba) for HSI classification. Specifically, a parallel lightweight Mamba and CNN block are developed to extract global and local spectral–spatial features. First, the cross-attention spectral–spatial Mamba module (CAS2MM) is proposed to leverage the global modeling of Mamba at linear complexity. In this module, dynamic positional embedding (DPE) is designed to enhance the spatial location information of visual sequences. The lightweight spectral–spatial Mamba blocks comprise an efficient scanning strategy and a lightweight Mamba design to efficiently extract global spectral–spatial features. And the cross-attention spectral–spatial fusion (CAS2F) is designed to learn cross correlation and fuse spectral–spatial features. Second, the lightweight spectral–spatial residual convolution module is proposed with lightweight spectral and spatial branches to extract local spectral–spatial features through residual learning. Finally, the adaptive global–local fusion is proposed to dynamically combine global Mamba features and local convolution features for a global–local spectral–spatial representation. Compared with state-of-the-art HSI classification methods, experimental results demonstrate that DualMamba achieves significant classification accuracy on three public HSI datasets and a superior reduction in model parameters and floating-point operations (FLOPs).