Matthew Vaughan, Samer Mheissen, Martyn Cobourne, Farooq Ahmed
{"title":"Diagnostic accuracy of artificial intelligence for dental and occlusal parameters using standardized clinical photographs.","authors":"Matthew Vaughan, Samer Mheissen, Martyn Cobourne, Farooq Ahmed","doi":"10.1016/j.ajodo.2025.01.017","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>SmileMate (SmileMate, Dental Monitoring SAS, Paris, France) is an artificial intelligence (AI)-based Web site that uses intraoral photographs to assess patients' dental and orthodontic parameters and provide a report. This study aimed to investigate the ability of an AI assessment tool (SmileMate) for orthodontic and dental parameters.</p><p><strong>Methods: </strong>A United Kingdom-based prospective clinical study enrolled 35 participants in the study. The participants' occlusal and dental parameters were assessed, and standardized orthodontic photographs were taken and uploaded to the SmileMate Web site to produce an AI-generated assessment. A total of 19 parameters were evaluated: 9 orthodontic parameters and 10 dental parameters covering both soft and hard tissues. A crosstabulation for AI and clinician assessments was reported using Fisher exact tests. Cohen's kappa was calculated to provide an agreement between the gold standard (clinician assessment) and SmileMate (AI assessment). Finally, the sensitivity, specificity, and area under the curve were calculated.</p><p><strong>Results: </strong>Statistically significant differences between a direct in-person assessment and the SmileMate AI assessment were noted across 9 of the 19 parameters (P <0.05, Fisher exact test). The overall kappa value was fair (0.29), with a variety of agreements between AI and clinician assessments; the level of agreement ranged from poor in 2 parameters (lateral open bite and teeth fracture) to almost perfect for missing and retained teeth. The level of agreement ranged from slight to moderate for the other variables in this study. The overall sensitivity of the AI-generated assessments was 72%, and the specificity was 54%. The specificity of AI was very low for gingivitis and oral hygiene, indicating a very high probability of false-positive findings for those parameters.</p><p><strong>Conclusions: </strong>The overall agreement between SmileMate and the clinician's assessment was slight to moderate. AI-generated assessments are inadequate for evaluating malocclusion.</p>","PeriodicalId":50806,"journal":{"name":"American Journal of Orthodontics and Dentofacial Orthopedics","volume":" ","pages":""},"PeriodicalIF":2.7000,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Journal of Orthodontics and Dentofacial Orthopedics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.ajodo.2025.01.017","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"DENTISTRY, ORAL SURGERY & MEDICINE","Score":null,"Total":0}
引用次数: 0
Abstract
Introduction: SmileMate (SmileMate, Dental Monitoring SAS, Paris, France) is an artificial intelligence (AI)-based Web site that uses intraoral photographs to assess patients' dental and orthodontic parameters and provide a report. This study aimed to investigate the ability of an AI assessment tool (SmileMate) for orthodontic and dental parameters.
Methods: A United Kingdom-based prospective clinical study enrolled 35 participants in the study. The participants' occlusal and dental parameters were assessed, and standardized orthodontic photographs were taken and uploaded to the SmileMate Web site to produce an AI-generated assessment. A total of 19 parameters were evaluated: 9 orthodontic parameters and 10 dental parameters covering both soft and hard tissues. A crosstabulation for AI and clinician assessments was reported using Fisher exact tests. Cohen's kappa was calculated to provide an agreement between the gold standard (clinician assessment) and SmileMate (AI assessment). Finally, the sensitivity, specificity, and area under the curve were calculated.
Results: Statistically significant differences between a direct in-person assessment and the SmileMate AI assessment were noted across 9 of the 19 parameters (P <0.05, Fisher exact test). The overall kappa value was fair (0.29), with a variety of agreements between AI and clinician assessments; the level of agreement ranged from poor in 2 parameters (lateral open bite and teeth fracture) to almost perfect for missing and retained teeth. The level of agreement ranged from slight to moderate for the other variables in this study. The overall sensitivity of the AI-generated assessments was 72%, and the specificity was 54%. The specificity of AI was very low for gingivitis and oral hygiene, indicating a very high probability of false-positive findings for those parameters.
Conclusions: The overall agreement between SmileMate and the clinician's assessment was slight to moderate. AI-generated assessments are inadequate for evaluating malocclusion.
期刊介绍：
Published for more than 100 years, the American Journal of Orthodontics and Dentofacial Orthopedics remains the leading orthodontic resource. It is the official publication of the American Association of Orthodontists, its constituent societies, the American Board of Orthodontics, and the College of Diplomates of the American Board of Orthodontics. Each month its readers have access to original peer-reviewed articles that examine all phases of orthodontic treatment. Illustrated throughout, the publication includes tables, color photographs, and statistical data. Coverage includes successful diagnostic procedures, imaging techniques, bracket and archwire materials, extraction and impaction concerns, orthognathic surgery, TMJ disorders, removable appliances, and adult therapy.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
