Hierarchical vision transformer model for polyp segmentation

Medical image analysis plays a powerful role in clinical assistance for the diagnosis and treatment of diseases. Image segmentation is an essential part of the medical imaging process as it extracts the region of interest through semi-automated or automated methods. Deep learning approaches have emerged as a fast-growing research field in medical image analysis. Vision transformers (ViT) are deep learning models that came up as a competing substitute for convolutional neural networks. ViT reports breakthroughs in computer vision tasks including object classification, detection, localization, and segmentation. Colon polyp detection and segmentation is a challenging task in the medical diagnosis and prognosis of colorectal cancer. Early detection and segmentation of polyp regions are of the utmost importance in preventing disease in later stages. In this work, we explore a hierarchical vision transformer as the backbone, replacing convolutional neural networks (CNNs) for the segmentation of polyps. The hierarchical vision transformer is composed of several stages, each having a different resolution. Through the use of a convolutional decoder, the patches from various stages are successively combined to produce full pre-dictions. The transformer backbone has a global receptive field at every stage that provide finer-grained and globally relevant predictions. Experimental results indicate that we can fine-tune the architecture to generate promising results on segmentation metrics even on smaller datasets, with mean Dice and mean IoU scores of 74% and 73% on the Kvasir-SEG dataset.