Diagnostic accuracy of artificial intelligence for dental and occlusal parameters using standardized clinical photographs

Introduction

SmileMate (SmileMate, Dental Monitoring SAS, Paris, France) is an artificial intelligence (AI)-based Web site that uses intraoral photographs to assess patients’ dental and orthodontic parameters and provide a report. This study aimed to investigate the ability of an AI assessment tool (SmileMate) for orthodontic and dental parameters.

Methods

A United Kingdom-based prospective clinical study enrolled 35 participants in the study. The participants’ occlusal and dental parameters were assessed, and standardized orthodontic photographs were taken and uploaded to the SmileMate Web site to produce an AI-generated assessment. A total of 19 parameters were evaluated: 9 orthodontic parameters and 10 dental parameters covering both soft and hard tissues. A crosstabulation for AI and clinician assessments was reported using Fisher exact tests. Cohen’s kappa was calculated to provide an agreement between the gold standard (clinician assessment) and SmileMate (AI assessment). Finally, the sensitivity, specificity, and area under the curve were calculated.

Results

Statistically significant differences between a direct in-person assessment and the SmileMate AI assessment were noted across 9 of the 19 parameters (P <0.05, Fisher exact test). The overall kappa value was fair (0.29), with a variety of agreements between AI and clinician assessments; the level of agreement ranged from poor in 2 parameters (lateral open bite and teeth fracture) to almost perfect for missing and retained teeth. The level of agreement ranged from slight to moderate for the other variables in this study. The overall sensitivity of the AI-generated assessments was 72%, and the specificity was 54%. The specificity of AI was very low for gingivitis and oral hygiene, indicating a very high probability of false-positive findings for those parameters.

Conclusions

The overall agreement between SmileMate and the clinician’s assessment was slight to moderate. AI-generated assessments are inadequate for evaluating malocclusion.