Dual-teacher self-distillation registration for multi-modality medical image fusion

Abstract

Misaligned multimodal medical images pose challenges to the fusion task, resulting in structural distortions and edge artifacts in the fusion results. Existing registration networks primarily consider single-scale deformation fields at each stage, thereby neglecting long-range connections between non-adjacent stages. Moreover, in the fusion task, due to the quadratic computational complexity faced by Transformers during feature extraction, they are unable to effectively capture long-range correlated features. To address these problems, we propose an image registration and fusion method called DTMFusion. DTMFusion comprises two main networks: a Dual-Teacher Self-Distillation Registration (DTSDR) network and a Mamba-Conv-based Fusion (MCF) network. The registration network employs a pyramid progressive architecture to generate independent deformation fields at each layer. We introduce a dual-teacher self-distillation scheme that leverages past learning history and the current network structure as teacher guidance to constrain the generated deformation fields. For the fusion network, we introduced Mamba to address the quadratic complexity problem of Transformers. Specifically, the fusion network involves two key components: the Shallow Fusion Module (SFM) and the Cross-Modality Fusion Module (CFM). The SFM achieves lightweight cross-modality interaction through channel exchange, while the CFM leverages inherent cross-modality relationships to enhance the representation capability of fusion results. Through the collaborative effort of these components, the network can effectively integrate cross-modality complementary information and maintain appropriate apparent strength from a global perspective. Extensive experimental analysis demonstrates the superiority of this method in fusing misaligned medical images.