Mélissa Peters, Maxime Le Clercq, Antoine Yanni, Xavier Vanden Eynden, Lalmand Martin, Noémie Vanden Haute, Szonja Tancredi, Céline De Passe, Edward Boutremans, Jerome Lechien, Didier Dequanter
{"title":"ChatGPT and trainee performances in the management of maxillofacial patients.","authors":"Mélissa Peters, Maxime Le Clercq, Antoine Yanni, Xavier Vanden Eynden, Lalmand Martin, Noémie Vanden Haute, Szonja Tancredi, Céline De Passe, Edward Boutremans, Jerome Lechien, Didier Dequanter","doi":"10.1016/j.jormas.2024.102090","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>ChatGPT is an artificial intelligence based large language model with the ability to generate human-like response to text input, its performance has already been the subject of several studies in different fields. The aim of this study was to evaluate the performance of ChatGPT in the management of maxillofacial clinical cases.</p><p><strong>Materials and methods: </strong>A total of 38 clinical cases consulting at the Stomatology-Maxillofacial Surgery Department were prospectively recruited and presented to ChatGPT, which was interrogated for diagnosis, differential diagnosis, management and treatment. The performance of trainees and ChatGPT was compared by three blinded board-certified maxillofacial surgeons using the AIPI score.</p><p><strong>Results: </strong>The average total AIPI score assigned to the practitioners was 18.71 and 16.39 to ChatGPT, significantly lower (p < 0.001). According to the experts, ChatGPT was significantly less effective for diagnosis and treatment (p < 0.001). Following two of the three experts, ChatGPT was significantly less effective in considering patient data (p = 0.001) and suggesting additional examinations (p < 0.0001). The primary diagnosis proposed by ChatGPT was judged by the experts as not plausible and /or incomplete in 2.63 % to 18 % of the cases, the additional examinations were associated with inadequate examinations in 2.63 %, to 21.05 % of the cases and proposed an association of pertinent, but incomplete therapeutic findings in 18.42 % to 47.37 % of the cases, while the therapeutic findings were considered pertinent, necessary and inadequate in 18.42 % of cases.</p><p><strong>Conclusions: </strong>ChatGPT appears less efficient in diagnosis, the selection of the most adequate additional examination and the proposition of pertinent and necessary therapeutic approaches.</p>","PeriodicalId":56038,"journal":{"name":"Journal of Stomatology Oral and Maxillofacial Surgery","volume":" ","pages":"102090"},"PeriodicalIF":2.2000,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Stomatology Oral and Maxillofacial Surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.jormas.2024.102090","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Dentistry","Score":null,"Total":0}
引用次数: 0
Abstract
Introduction: ChatGPT is an artificial intelligence based large language model with the ability to generate human-like response to text input, its performance has already been the subject of several studies in different fields. The aim of this study was to evaluate the performance of ChatGPT in the management of maxillofacial clinical cases.
Materials and methods: A total of 38 clinical cases consulting at the Stomatology-Maxillofacial Surgery Department were prospectively recruited and presented to ChatGPT, which was interrogated for diagnosis, differential diagnosis, management and treatment. The performance of trainees and ChatGPT was compared by three blinded board-certified maxillofacial surgeons using the AIPI score.
Results: The average total AIPI score assigned to the practitioners was 18.71 and 16.39 to ChatGPT, significantly lower (p < 0.001). According to the experts, ChatGPT was significantly less effective for diagnosis and treatment (p < 0.001). Following two of the three experts, ChatGPT was significantly less effective in considering patient data (p = 0.001) and suggesting additional examinations (p < 0.0001). The primary diagnosis proposed by ChatGPT was judged by the experts as not plausible and /or incomplete in 2.63 % to 18 % of the cases, the additional examinations were associated with inadequate examinations in 2.63 %, to 21.05 % of the cases and proposed an association of pertinent, but incomplete therapeutic findings in 18.42 % to 47.37 % of the cases, while the therapeutic findings were considered pertinent, necessary and inadequate in 18.42 % of cases.
Conclusions: ChatGPT appears less efficient in diagnosis, the selection of the most adequate additional examination and the proposition of pertinent and necessary therapeutic approaches.
期刊介绍：
J Stomatol Oral Maxillofac Surg publishes research papers and techniques - (guest) editorials, original articles, reviews, technical notes, case reports, images, letters to the editor, guidelines - dedicated to enhancing surgical expertise in all fields relevant to oral and maxillofacial surgery: from plastic and reconstructive surgery of the face, oral surgery and medicine, … to dentofacial and maxillofacial orthopedics.
Original articles include clinical or laboratory investigations and clinical or equipment reports. Reviews include narrative reviews, systematic reviews and meta-analyses.
All manuscripts submitted to the journal are subjected to peer review by international experts, and must:
Be written in excellent English, clear and easy to understand, precise and concise;
Bring new, interesting, valid information - and improve clinical care or guide future research;
Be solely the work of the author(s) stated;
Not have been previously published elsewhere and not be under consideration by another journal;
Be in accordance with the journal''s Guide for Authors'' instructions: manuscripts that fail to comply with these rules may be returned to the authors without being reviewed.
Under no circumstances does the journal guarantee publication before the editorial board makes its final decision.
The journal is indexed in the main international databases and is accessible worldwide through the ScienceDirect and ClinicalKey Platforms.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
