SkinFormer: Robust Vision Transformer for Automatic Skin Disease Identification

Abstract

The largest, most visible, and most exposed organ of the human body is the skin. Skin diseases impact about a third of the global population. Furthermore, many serious skin diseases, such as melanoma, may remain misdiagnosed for years since only experienced dermatologists can reliably diagnose them. As a result, some regions and demographic groups may be more negatively impacted than others in terms of access to such medical professionals. Given the range and possible hazards of skin diseases, democratizing access to accurate identification is critical. Fortunately, automated deep-learning systems have made significant advances in picture classification in recent years. In this paper we present SkinFormer, a vision transformer trained using strong augmentations and optimization techniques to ensure robustness and generalization. The dataset we use is a combination of 3 different datasets from previous works, and contains a total of 48,322 images. The choice of datasets and augmentations ensures strong generalization even on consumergrade hardware. We achieve a top-1 accuracy of 84.43% and a top-5 accuracy of 93.89%. We publish pre-trained model weights under an open source license.