CH-Net: A Cross Hybrid Network for Medical Image Segmentation

Abstract

Accurate and automated segmentation of medical images plays a crucial role in diagnostic evaluation and treatment planning. In recent years, hybrid models have gained considerable popularity in diverse medical image segmentation tasks, as they leverage the benefits of both convolution and self-attention to capture local and global dependencies simultaneously. However, most existing hybrid models treat convolution and self-attention as independent components and integrate them using simple fusion methods, neglecting the potential complementary information between their weight allocation mechanisms. To address this issue, we propose a cross hybrid network (CH-Net) for medical image segmentation, in which convolution and self-attention are hybridized in a cross-collaborative manner. Specifically, we introduce a cross hybrid module (CHM) between the parallel convolution layer and self-attention layer in each building block of CH-Net. This module extracts attention with distinct dimensional information from convolution and self-attention, respectively, and uses this complementary information to enhance the feature representation of both components. In contrast to the traditional approach where each module learned independently, the CHM facilitates the interactive learning of complementary information between convolutional layer and self-attention layer, which significantly enhances the segmentation capabilities of the model. The superiority of our approach over various hybrid models is demonstrated through experimental evaluations conducted on three publicly available benchmarks: ACDC, synapse, and EM.