Assessing diagnostic performance for common skin diseases using an AI-assisted tele-expertise platform: a proof of concept.

Abstract

Advancements in machine learning (ML) are making artificial intelligence more feasible in dermatology, with promising results for diagnosing skin cancers, though few studies cover common or inflammatory dermatoses. To evaluate the diagnostic accuracy for common non-cancerous skin diseases and the clinical applicability of an ML model in practical telemedicine. A prospective, multi-centre, diagnostic accuracy study including patients with common dermatoses, between October 2022 and July 2023, was performed. The top three diagnoses (Top 1, Top 2 and Top 3) from the AI system, trained to recognize 25 common dermatoses based on skin lesion images and medical data, were compared to diagnoses by two dermatologists (gold standard) to calculate the AI model's diagnostic accuracy, sensitivity, and specificity. Two versions of the AI software were evaluated: version 1 (V1) and version 2 (V2) with and without medical supervision (MS), referring to the use of metadata to control diagnostic predictions. Seventy participants and 195 photographs were included. The sensitivity and specificity of the Top 3 algorithm were 88% and 90%, respectively, for V2, with a significant improvement compared with V1. For V1, diagnostic accuracy was 0.57 (0.46;0.69) for Top 1, 0.70 (0.59;0.81) for Top 2, and 0.81 (0.72;0.91) for Top 3. For V2, diagnostic accuracy was 0.69 (0.58;0.79) and 0.71 (0.61;0.82) without and with MS, respectively, for Top 1; 0.87 (0.79;0.95) for Top 2; and 0.90 (0.83;0.97) for Top 3. Our AI model appears to be a promising tool for triaging and diagnosing skin lesions, especially for non-specialist physicians.