SwinEff-AttentionNet: a dual hybrid model for breast image segmentation and classification using multiple ultrasound modality

Abstract

Breast cancer is the most prevalent malignancy among women globally, with early detection playing a pivotal role in improving survival rates. However, ultrasound image interpretation remains a challenge due to noise, indistinct lesion boundaries, and the need for skilled radiologists, especially in resource-limited settings. This research introduces SwinEff-AttentionNet, a novel hybrid deep learning framework combining Swin transformers, EfficientNet layers, and Efficient Local Self-Attention (ELSA) modules to enhance breast ultrasound image analysis. Utilizing hierarchical feature extraction, the proposed architecture excels in segmentation and classification tasks. It was evaluated on two benchmark datasets: BUSI and Breast-Lesions-USG. For classification, SwinEff-AttentionNet achieved an accuracy of 98.50% and 95.84% on the BUSI and Breast-Lesions-USG datasets, respectively, outperforming state-of-the-art models such as ViT, DeiT, PVT, CrossViT and CvT. Similarly, segmentation performance yielded Dice scores of 92% and 87.82%, IoU scores of 88.7% and 83%, and AUC values of 91.38% and 89.72% on the BUSI and Breast-Lesions-USG datasets, respectively, underscoring its robustness across diverse imaging conditions. The dual-task nature of SwinEff-AttentionNet demonstrates its versatility, offering clinicians a reliable tool for both lesion localization and diagnosis. This study highlights the potential of advanced hybrid architectures in addressing the limitations of traditional imaging frameworks, paving the way for improved diagnostic accuracy and clinical decision-making in breast cancer care.