A Hybrid Medical Image Semantic Segmentation Network Based on Novel Mamba and Transformer

Abstract

Recently, deep learning has greatly advanced medical image segmentation. Convolutional neural networks (CNNs) excel in capturing local image features, whereas ViT adeptly models long-range dependencies through multi-head self-attention mechanisms. Despite their strengths, both CNN and ViT face challenges in efficiently processing long-range dependencies in medical images and often require substantial computational resources. To address this, we propose a novel hybrid model combining Mamba and Transformer architectures. Our model integrates ViT's self-attention modules within a pure-vision Mamba U-shaped encoder, capturing both global and local information through nested Transformer and Mamba modules. Additionally, a multi-scale feed-forward neural network is incorporated within the Mamba blocks to enhance feature diversity by capturing fine-grained local details. Finally, a channel-adaptive feature (CAF) fusion module is introduced at the original skip connections to mitigate feature loss during information fusion and to improve segmentation accuracy in boundary regions. Quantitative and qualitative experiments were conducted on two public datasets: breast ultrasound image (BUSI) and clinic. The Dice score, Intersection over Union (IoU) score, recall score, F1 score and 95% Hausdorff distance (HD95) of the proposed model on the BUSI dataset were 0.7918, 0.7016, 0.8508, 0.7919 and 12.04 mm, respectively. On ClinicDB, these metrics reach 0.9239, 0.8671, 0.9278, 0.9239 and 5.49 mm, respectively. The proposed model outperforms existing state-of-the-art CNN-, Transformer- and Mamba-based methods in segmentation accuracy, according to experimental data.