ResTransUNet: A hybrid CNN-transformer approach for liver and tumor segmentation in CT images

Abstract

Background and Objective:

Accurate medical tumor segmentation is critical for early diagnosis and treatment planning, significantly improving patient outcomes. This study aims to enhance liver and tumor segmentation from CT and liver images by developing a novel model, ResTransUNet, which combines convolutional and transformer blocks to improve segmentation accuracy.

Methods:

The proposed ResTransUNet model is a custom implementation inspired by the TransUNet architecture, featuring a Standalone Transformer Block and ResNet50 as the backbone for the encoder. The hybrid architecture leverages the strengths of Convolutional Neural Networks (CNNs) and Transformer blocks to capture both local features and global context effectively. The encoder utilizes a pre-trained ResNet50 to extract rich hierarchical features, with key feature maps to preserved it as skip connections. The Standalone Transformer Block, integrated into the model, employs multi-head attention mechanisms to capture long-range dependencies across the image, enhancing segmentation performance in complex cases. The decoder reconstructs the segmentation mask by progressively upsampling encoded features while integrating skip connections, ensuring both semantic information and spatial details are retained. This process culminates in a precise binary segmentation mask that effectively distinguishes liver and tumor regions.

Results:

The ResTransUNet model achieved superior Dice Similarity Coefficient (DSC) for liver segmentation (98.3% on LiTS and 98.4% on 3D-IRCADb-01) and for tumor segmentation from CT images (94.7% on LiTS and 89.8% on 3D-IRCADb-01) as well as from liver images (94.6% on LiTS and 91.1% on 3D-IRCADb-01). The model also demonstrated high precision, sensitivity, and specificity, outperforming current state-of-the-art methods in these tasks.

Conclusions:

The ResTransUNet model demonstrates robust and accurate performance in complex medical image segmentation tasks, particularly in liver and tumor segmentation. These findings suggest that ResTransUNet has significant potential for improving the precision of surgical interventions and therapy planning in clinical settings.