SFMamba: a novel spatial-frequency collaborative learning for multimodal medical image fusion with mamba

Abstract

The objective of medical image fusion is to discern and amalgamate complementary features extracted from multimodal medical images, producing fused representations that are more interpretable and informative. This enhancement facilitates higher diagnostic accuracy and efficiency. While Transformer-based fusion methods excel at capturing long-range dependencies and enable parallel computation, their computational cost rises sharply with increasing input dimensions, often reaching quadratic complexity. To address this challenge, we propose a novel spatial-frequency domain fusion network, termed as SFMamba, which exploits a frequency transformation in conjunction with the Mamba model to fully leverage both spatial and frequency information. An efficient Mamba branch incorporates a spatial-frequency state-space (SFSS) model, reducing computational burden to linear or near-linear complexity. The selective-band feature extraction (SBFE) branch is constructed using a discrete wavelet pyramid, designed to capture multi-scale frequency components consistently across source images. To dynamically and effectively fuse multi-modal complementary information, we introduce a multi-domain feature fusion (MDFF) module that elevates fusion performance. Training is conducted with a multi-teacher learning strategy (MTLS) that integrates pre-trained convolutional neural networks and transformer-based fusion methods to generate multiple pseudo-labels, guiding the network to inherit fused knowledge from these priors. Extensive experiments demonstrate that SFMamba achieves state-of-the-art performance in both subjective and objective evaluations. The code for SFMamba is available at https://github.com/DZSYUNNAN/SFMamba.