TransMixer: A Hybrid Transformer and CNN Architecture for Polyp Segmentation

Abstract

Learning how to fully extract global representations and local features is a key factor in improving the performance of polyp segmentation. In this paper, we explore the potential of combined techniques of Transformers and convolutional neural networks (CNNs) to address the challenges of polyp segmentation. Specifically, we present TransMixer, a hybrid interaction fusion architecture of the Transformer branch and the CNN branch, which is able to enhance the local details of global representations and the global context awareness of local features. To achieve this, we first bridge the semantic gap between the Transformer branch and the CNN branch through the Interaction Fusion Module (IFM), and then make full use of both respective properties to enhance polyp feature representations. After that, we further propose the Hierarchical Attention Module (HAM) to collect polyp semantic information from high-level features to gradually guide the recovery of polyp spatial information in low-level features. Quantitative and qualitative results show that the proposed model is more robust to various complex situations compared to existing methods, and achieves state-of-the-art performance in polyp segmentation.