SMAFusion: Multimodal medical image fusion based on spatial registration and local-global multi-scale feature adaptive fusion

Abstract

Aiming at the current image fusion methods relying entirely on paired images after registration, the fusion efficiency is low, and the existing image registration methods overlook the impact of image distribution differences on registration outcomes, as well as the poor capability of the image fusion methods for local-global multi-scale feature extraction from source images during image fusion, we propose a multimodal medical image fusion method based on spatial registration and local-global multi-scale feature adaptive fusion (SMAFusion). In SMAFusion, the spatial registration module is utilized to minimize the impact of image distribution differences on the registration results. Meanwhile, a local-global multi-scale feature encoder is proposed to fully extract local and global information at different scales. And an adaptive fusion strategy is employed to fuse the multi-scale features of different modal images. The experimental results show that this method has an average improvement of 24 %, 121.04 %, 19.57 %, 40.36 %, and 97.62 % compared to the eight comparative methods in five evaluation indicators: mutual information (MI), difference correlation sum (SCD), visual fidelity (VIFF), gradient fusion performance (QAB/F) and structural similarity (SSIM), respectively. The fused image preserves more details and texture information, and exhibits high consistency with the original image in terms of overall structure while enhancing the fusion ability for unregistered images. The significant improvement in MI and SCD indicates that the fused image better integrates complementary information from multiple modalities, helping doctors obtain more diagnostic clues from a single image and reducing the risk of missed or incorrect diagnoses. The improvements in VIFF and SSIM suggest that the fused image performs better in structure preservation and visual clarity, making lesion areas and anatomical structures clearer, thereby enhancing clinical readability and diagnostic efficiency. The enhancement in QAB/F further demonstrates the ability to preserve image edges and texture details, which is directly beneficial for observing fine structures.