Assessing the Performance of ChatGPT 3.5 and ChatGPT 4 in Operative Dentistry and Endodontics: An Exploratory Study

IF 4.3 Q1 PSYCHOLOGY, MULTIDISCIPLINARY

Human Behavior and Emerging Technologies Pub Date : 2024-10-08 DOI:10.1155/2024/1119816

Niher Tabassum Snigdha, Rumesa Batul, Mohmed Isaqali Karobari, Abdul Habeeb Adil, Ali Azhar Dawasaz, Mohammad Shahul Hameed, Vini Mehta, Tahir Yusuf Noorani

{"title":"Assessing the Performance of ChatGPT 3.5 and ChatGPT 4 in Operative Dentistry and Endodontics: An Exploratory Study","authors":"Niher Tabassum Snigdha, Rumesa Batul, Mohmed Isaqali Karobari, Abdul Habeeb Adil, Ali Azhar Dawasaz, Mohammad Shahul Hameed, Vini Mehta, Tahir Yusuf Noorani","doi":"10.1155/2024/1119816","DOIUrl":null,"url":null,"abstract":"Background: Artificial intelligence is an innovative technology that mimics human cognitive capacities and has gathered the world’s attention through its vast applications in various fields.Aim: This study is aimed at assessing the effects of ChatGPT 3.5 and ChatGPT 4 on the validity, reliability, and authenticity of standard assessment techniques used in undergraduate dentistry education.Methodology: Twenty questions, each requiring a single best answer, were selected from two domains: 10 from operative dentistry and 10 from endodontics. These questions were divided equally, with half presented with multiple choice options and the other half without. Two investigators used different ChatGPT accounts to generate answers, repeating each question three times. The answers were scored between 0% and 100% based on their accuracy. The mean score of the three attempts was recorded, and statistical analysis was conducted.Results: No statistically significant differences were found between ChatGPT 3.5 and ChatGPT 4 in the accuracy of their responses. Additionally, the analysis showed high consistency between the two reviewers, with no significant difference in their assessments.Conclusion: This study evaluated the performance of ChatGPT 3.5 and ChatGPT 4 in answering questions related to endodontics and operative dentistry. The results showed no statistically significant differences between the two versions, indicating comparable response accuracy. The consistency between reviewers further validated the reliability of the assessment process.","PeriodicalId":36408,"journal":{"name":"Human Behavior and Emerging Technologies","volume":null,"pages":null},"PeriodicalIF":4.3000,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1155/2024/1119816","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Human Behavior and Emerging Technologies","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1155/2024/1119816","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

Background: Artificial intelligence is an innovative technology that mimics human cognitive capacities and has gathered the world’s attention through its vast applications in various fields.

Aim: This study is aimed at assessing the effects of ChatGPT 3.5 and ChatGPT 4 on the validity, reliability, and authenticity of standard assessment techniques used in undergraduate dentistry education.

Methodology: Twenty questions, each requiring a single best answer, were selected from two domains: 10 from operative dentistry and 10 from endodontics. These questions were divided equally, with half presented with multiple choice options and the other half without. Two investigators used different ChatGPT accounts to generate answers, repeating each question three times. The answers were scored between 0% and 100% based on their accuracy. The mean score of the three attempts was recorded, and statistical analysis was conducted.

Results: No statistically significant differences were found between ChatGPT 3.5 and ChatGPT 4 in the accuracy of their responses. Additionally, the analysis showed high consistency between the two reviewers, with no significant difference in their assessments.

Conclusion: This study evaluated the performance of ChatGPT 3.5 and ChatGPT 4 in answering questions related to endodontics and operative dentistry. The results showed no statistically significant differences between the two versions, indicating comparable response accuracy. The consistency between reviewers further validated the reliability of the assessment process.

Abstract Image

查看原文本刊更多论文

评估 ChatGPT 3.5 和 ChatGPT 4 在牙科手术和根管治疗中的性能：一项探索性研究

背景：目的：本研究旨在评估 ChatGPT 3.5 和 ChatGPT 4 对本科口腔医学教育中使用的标准评估技术的有效性、可靠性和真实性的影响：从两个领域中选择了 20 个问题，每个问题都有一个最佳答案：其中 10 个问题来自牙科手术，10 个问题来自牙髓病学。这些问题被平均分配，一半有多项选择题，另一半没有。两名调查人员使用不同的 ChatGPT 账户生成答案，每个问题重复三次。根据答案的准确性在 0% 和 100% 之间打分。记录三次尝试的平均得分，并进行统计分析：结果：ChatGPT 3.5 和 ChatGPT 4 在回答的准确性方面没有发现明显的统计学差异。此外，分析表明两位审稿人之间的一致性很高，他们的评估结果没有明显差异：本研究评估了 ChatGPT 3.5 和 ChatGPT 4 在回答牙髓病学和牙科手术相关问题时的表现。结果表明，两个版本在统计学上没有明显差异，这表明回答的准确性相当。审阅者之间的一致性进一步验证了评估过程的可靠性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Human Behavior and Emerging Technologies Social Sciences-Social Sciences (all)

CiteScore

17.20

自引率

8.70%

发文量

期刊介绍： Human Behavior and Emerging Technologies is an interdisciplinary journal dedicated to publishing high-impact research that enhances understanding of the complex interactions between diverse human behavior and emerging digital technologies.