CM-CSAMFNet: A cross-modality channel and spatial attention module fusion network for multimodal medical image fusion

The fusion technology of functional and structural images contributes to clinical diagnosis by integrating complementary information from different modalities. However, traditional state-of-the-art fusion methods and convolutional networks still require manually designed fusion strategies, which are inefficient and struggle to effectively merge complementary information across modalities. In addition, multiscale fusion methods suffer from excessive model parameters and inadequate consideration of long-range dependencies. To overcome these limitations, an attention-based end-to-end framework (CM-CSAMFNet) is proposed for medical image fusion using a multiscale autoencoder architecture. To design a learnable fusion strategy, we introduce a convolutional block attention module fusion network (CBAMFNet), which leverages cross-modal channel and spatial attention mechanisms to replace conventional fusion approaches. To reduce the number of parameters in multiscale network, the entire network employs ghost convolution techniques, which require only a small number of convolutional operations while extensively utilizing linear computations. Furthermore, to fully account for long-range dependencies, a cross-domain attention mechanism named the cross-modal residual convolutional block attention module (RCBAM) is proposed. This mechanism aims to comprehensively integrate locally complementary features and enhance global brightness information. More specifically, the cross-domain attention module incorporates spatial and channel attention mechanisms to integrate long-range dependencies within and across different modalities. Compared to existing approaches, the proposed fusion algorithm achieves superior performance in SPECT–MRI and PET–MRI image fusion tasks, as evaluated by both subjective and objective metrics. The code of the proposed method is available at https://github.com/ahu-dsp/CM-CSAMFNet