Multimodal medical image fusion based on dilated convolution and attention-based graph convolutional network

Abstract

To address the limitations of current deep learning-based multimodal medical image fusion methods, which include insufficient representation of low- and high-level semantic information and weak robustness against noisy data, this paper proposes a multimodal medical image fusion method based on dilated convolution and attention-based graph convolutional networks, termed CGCN-Fusion. The framework comprises three main components: an encoder, a fusion module, and a decoder. The encoder integrates a low-level CNN encoder with a high-level GCN encoder to comprehensively enhance the representation of semantic features. Specifically, the low-level CNN encoder, through the introduction of dilated convolution technology, significantly enhances its ability to represent low-level semantic information, enabling the model to capture fine-grained details of images better. Meanwhile, the high-level GCN encoder leverages the unique strengths of graph convolutional networks to improve robustness against noise while integrating self-attention and multi-head attention mechanisms for a richer high-level semantic representation. The self-attention mechanism captures semantic information of key nodes, while the multi-head attention integrates structural encoding to model spatial-semantic relationships among nodes. The fusion module systematically integrates the extracted features, and the decoder reconstructs the fused image. Through comparisons with nine state-of-the-art methods, CGCN-Fusion demonstrates superior visual quality and objective performance, validating its effectiveness and clinical applicability in multimodal medical image fusion.