SpectMamba: Remote sensing change detection network integrating frequency and visual state space model

Abstract

In recent years, the fusion of Convolutional Neural Network (CNNs) and Transformer models, which can simultaneously leverage the former’s efficiency in local feature extraction and the latter’s advantage in capturing long-range dependencies, has achieved complementary strengths and demonstrated superior modeling potential. However, some of the high-frequency subtle changes and periodic structural changes (e.g., regularly arranged clusters of reconstructed buildings) in multispectral remote sensing images are often difficult to detect in the spatial domain; at the same time, the high computational complexity of the Transformer model restricts its practical application. Recently, the state-space model-based Mamba architecture has performed well in the RSCD task, efficiently learning image global information with linear complexity. Based on this, this study hypothesizes that a strategy combining spectral layers with visual state space (VSS) modules can more efficiently parse these challenges in dense prediction tasks. Specifically, we propose the frontier strategy of using a spectral layer for the initial layer and a VSS layer for the deeper layer and verify its effectiveness through extensive experiments. At the same time, we identify and optimize the limitations of VSS in independently processing the high-frequency information output from the spectral layer, and develop Conv-VSS. These techniques are integrated and extended into a network called SpectMamba, which fuses the spectral layer and Conv-VSS to more appropriately capture feature representations by analyzing the feature images in both the frequency domain and spatial features while avoiding the complexity associated with high-dimensional matrix operations in self-attention. Extensive experimental results on three publicly available datasets show that SpectMamba significantly outperforms existing techniques on several performance metrics.