DMD: Dual attention fusion and multi-scale feature fusion decoding for medical image segmentation

Abstract

High-precision medical image segmentation plays a decisive role in clinical diagnosis, treatment efficacy evaluation, and the development of personalized therapeutic strategies. Although models based on U-Net and its variants have achieved remarkable results across various segmentation tasks, traditional convolutional neural networks (CNNs) are inherently limited by their local receptive fields, restricting their ability to capture long-range dependencies and global contextual information. In contrast, Transformer architectures leverage self-attention mechanisms to model global information, demonstrating superior performance in segmenting complex anatomical structures. However, most existing Transformer-based segmentation frameworks fail to fully integrate multi-scale semantic features extracted by the encoder during the decoding stage, leading to suboptimal feature reconstruction and reduced segmentation accuracy. To address these issues, we propose a novel decoder framework—DMD. First, an Inverted Bottleneck CNN module is introduced in the decoding phase to efficiently capture and enhance the rich semantic features output by the encoder. Second, to resolve the semantic mismatch between encoder and decoder multi-scale features, a Dual Attention Fusion Module (DAFM) is designed, which employs a joint spatial-and-channel attention mechanism to achieve adaptive alignment of cross-scale semantic information. Finally, to mitigate spatial resolution degradation caused by single-scale upsampling, a Multi-scale Feature Fusion Module (MFFM) is constructed to enable deep integration and collaborative optimization of features at different semantic levels. Experimental results demonstrate that the proposed model achieves Dice scores of 82.93%, 91.32%, and 87.85% on the Synapse multi-organ segmentation, ACDC cardiac MRI segmentation, and Kvasir-SEG gastrointestinal endoscopy image segmentation datasets, respectively, outperforming existing methods and verifying the effectiveness and robustness of the framework in complex medical image segmentation tasks.