A Hybrid Vision Transformer Based Edge-Guided Boundary Attention U-Net for Skin Lesion Segmentation

Abstract

Accurate skin lesion segmentation plays a vital role in computer-aided diagnosis for early detection and timely treatment of skin cancer. However, existing deep learning models often struggle to simultaneously capture long-range contextual dependencies and preserve fine lesion boundary details in challenging dermoscopic images. To address these limitations, we propose Trans-EGBAU-Net, a hybrid Vision Transformer (ViT) based Edge-Guided Boundary Attention (EGBA) U-Net for accurate and boundary-aware skin lesion segmentation. The proposed framework employs a ViT encoder to learn global contextual representation from image patches through multiple transformer layers. To compensate for the limited local spatial sensitivity of transformer features, a lightweight Edge Attention Module (EAM) is incorporated after the encoder to enhance edge-aware representations around lesion boundaries. Furthermore, the proposed EGBA mechanism utilizes edge-enhanced transformer features generated by the EAM to guide the Boundary-Guided Attention Gate (BGAG) during skip-feature refinement, while a standard Attention Gate (AG) selectively emphasizes semantically relevant lesion regions. The complementary integration of AG and EGBA enables simultaneous semantic feature selection and contour-aware boundary refinement, resulting in improved delineation of irregular and low-contrast lesion boundaries. Experimental evaluation on the HAM10000, ISIC 2018, and PH2 datasets demonstrates competitive segmentation performance, achieving accuracies of 0.9793, 0.9688, and 0.9679; Dice coefficients of 0.9446, 0.9165, and 0.9155; IoU scores of 0.9009, 0.8753, and 0.8596; Sensitivities of 0.9796, 0.9345, and 0.9680; and Specificities of 0.9849, 0.9772, and 0.9488, respectively. Ablation studies further validate the effectiveness of the proposed attention mechanisms, making Trans-EGBAU-Net a reliable framework for boundary-aware skin lesion segmentation.