Assessment of a Smartphone-Based Neural Network Application for the Risk Assessment of Skin Lesions under Real-World Conditions.

Abstract

Introduction: The diagnostic performance of convolutional neural networks (CNNs) in diagnosing different types of skin cancer has been quite promising. Mobile phone applications with integrated artificial intelligence (AI) are an understudied area.

Objective: We evaluated the risk assessment of the SkinScreener (Medaia GmbH, Graz, Austria) AI-based algorithm in comparison with an expert panel of three dermatologists.

Methods: In this retrospective single-center study at the Department of Dermatology and Venereology in Graz, Austria. Photographs of lesions were taken by the users' mobile phone cameras. The algorithm allocated them to three risk classes. Blinded to AI's results, the images were evaluated by three dermatologists-our reference standard. A consensus was defined as at least a two-thirds majority.

Results: A total of 1,428 skin lesions were included. In 902 lesions (63.16%), there was full agreement, and in 441 lesions (30.88%) a two-thirds majority was reached. Eighty-five lesions (5.69%) had to be discussed in a joint review process. The tested algorithm reached a sensitivity of 76.9% (95% CI: 71.7%-81.5%) and a specificity of 80.9% (95% CI: 78.5%-83.2%). Overall accuracy results were 77.2%.

Conclusions: Our results indicate that the tested mobile phone algorithm is a valuable tool for the correct risk classification of various skin lesions. As expected, its performance is worse than in a professional setting. Nonetheless, the use of these applications on mobile phones should raise awareness of skin cancer and encourage users to deal more intensively with preventive measures. In light of our results, these applications are also reliable for use by non-professionals.