Modified Transformer-Based Pixel Segmentation for Breast Tumor Detection

Abstract

This study introduces a novel hybrid deep learning model that combines residual convolutional networks and a multilayer perceptron (MLP)-based transformer for precise breast lesion segmentation and classification using mammogram images. Initially, mammograms undergo preprocessing involving thresholding and Gabor-based pixel segmentation to extract informative patches. The proposed model leverages deep features extracted via convolutional neural networks, which are subsequently processed through self-attention and cross-attention mechanisms in a modified transformer architecture to capture both local and global dependencies for classification. The approach is rigorously evaluated on the publicly available INbreast dataset, achieving classification accuracies of 98.17% for a three-class (normal, benign, malignant) scenario and 96.74% for a more detailed five-class classification. The model demonstrates strong capabilities in differentiating subtle variations between malignant and benign tissues. These promising results suggest significant potential for practical clinical implementation, assisting radiologists by providing highly accurate diagnostic insights. Notably, this approach contributes substantially to automated breast cancer diagnostics, highlighting the efficacy of integrating convolutional neural network features with transformer architectures for improved segmentation and classification outcomes.