A novel approach to skin lesion segmentation using transformer attention and focal modulation

Precise segmentation of skin lesions is essential for early diagnosis of melanoma, yet it remains a complex task due to inconsistencies in image appearance caused by different clinical environments and patient-specific factors. To tackle this challenge, we introduce TAFM-Net, a novel deep learning framework that combines Transformer based Attention (TA) and Focal Modulation (FM) for enhanced lesion segmentation. The architecture employs an EfficientNetV2-B1 encoder to capture rich spatial and channel-wise features using TA, while FM is incorporated into the decoder’s skip connections to strengthen contextual learning and feature representation. Additionally, a dynamic loss function is proposed to balance region-level accuracy and boundary precision during training. Our method achieves Jaccard scores of 93.64%, 86.88%, and 92.88% on the ISIC 2016, 2017, and 2018 datasets, respectively, confirming its effectiveness and suitability for real-world dermatological applications.