Unified ARP-ViT-CNN system: Hybrid deep learning approach for segmenting and classifying multiple skin cancer lesions

Abstract

Skin cancer continues to be one of the most difficult conditions to diagnose accurately, as different lesion types can look very similar, and the images used for diagnosis often contain noise and large variations. Many existing deep learning models, such as traditional Neural Networks and Transformer-based models, often struggle to capture both the fine-grained local details and the broader context of the image. To address these challenges, we developed a hybrid ARP-ViT-CNN model that employs a three-stream feature extraction process, where Angular Radial Partitioning (ARP) extracts geometric and structural patterns, Convolutional Neural Networks (CNN) extract subtle local features, and exploits Vision Transformers (ViT) to code long-range dependencies through self-attention mechanisms. Then, use this feature representation to develop a more advanced classification model that can differentiate several forms of skin malignancies. The Proposed model integrates a comprehensive Pre-processing pipeline, including balancing and enhancing the dataset, to address the problem of underrepresented classes and image variability. We evaluated our model on the publicly available HAM10000 dataset for multi-class skin cancer classification and segmentation. As anticipated, the ARP-ViT-CNN model beat the conventional CNN-only and Transformer-only models on the dataset, achieving an overall accuracy of 98.2% with a precision of 0.94, a recall of 0.96, and a macro-F1 score of 0.95. The ARP-ViT-CNN segmentation module for outlining lesion boundaries performed well, and was especially reliable in providing accurate boundary delineation in the presence of the complicated lesions in the test data. The overall results indicate that our ARP-ViT-CNN framework is effective, and with the ViT being suitable for global contextual learning, ARP being the mechanism by which we were making the model invariant to rotation and scale features, and this is likely contributing to the model’s tolerance to complex dermatological expressions. The ARP-ViT-CNN model has established itself as a modern approach to fully automated skin cancer imaging for automated diagnosis and a basis for future AI-powered medical imaging.