Can artificial intelligence accurately detect and summarize anatomy education literature? A comparative analysis of ChatGPT and ScholarGPT

Q3 Medicine

Morphologie Pub Date : 2025-08-12 DOI:10.1016/j.morpho.2025.101061

D. Chytas , S. Kanakaris , M. Piagkou , I. Chryssanthou , A.V. Vasiliadis , K. Natsis

{"title":"Can artificial intelligence accurately detect and summarize anatomy education literature? A comparative analysis of ChatGPT and ScholarGPT","authors":"D. Chytas , S. Kanakaris , M. Piagkou , I. Chryssanthou , A.V. Vasiliadis , K. Natsis","doi":"10.1016/j.morpho.2025.101061","DOIUrl":null,"url":null,"abstract":"<div><h3>Purpose</h3><div>Artificial intelligence platforms have been suggested as tools that can facilitate anatomy teachers’ work and students’ learning process. We aimed to investigate the ability of ChatGPT to detect and summarize studies of the anatomy education literature compared to ScholarGPT, a version of ChatGPT specified in academic research. Secondly, we aimed to explore if the ability of each platform is influenced by the level of queries complexity.</div></div><div><h3>Methods</h3><div>We asked the two platforms to list five studies about each of the following three topics: (1) use of virtual reality in anatomy education, (2) use of stereoscopic virtual reality in anatomy education, (3) use of stereoscopic virtual reality in anatomy education, involving user's interaction with the virtual environment. We assessed if the retrieved studies fulfilled the search criteria, and if their summaries were accurate (if they contained true information about all the educational results of the article's abstract).</div></div><div><h3>Results</h3><div>The ChatGPT's percentages of successful detection were 100%, 60% and 0% respectively for the three queries. The percentages of accurate summaries were 60%, 20% and 0% respectively. ScholarGPT performed better, with a percentage of successful detection 100%, 60% and 40% respectively. The percentages of accurate summaries were 80%, 60% and 40% respectively. Both platforms showed bias in favor of the educational intervention.</div></div><div><h3>Conclusions</h3><div>ChatGPT and ScholarGPT are not currently at an adequate level to essentially aid researchers to detect and summarize studies of the anatomy education literature. Ongoing research may increase the ability of those platforms to provide more reliable information.</div></div>","PeriodicalId":39316,"journal":{"name":"Morphologie","volume":"109 367","pages":"Article 101061"},"PeriodicalIF":0.0000,"publicationDate":"2025-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Morphologie","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1286011525001134","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Medicine","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose

Artificial intelligence platforms have been suggested as tools that can facilitate anatomy teachers’ work and students’ learning process. We aimed to investigate the ability of ChatGPT to detect and summarize studies of the anatomy education literature compared to ScholarGPT, a version of ChatGPT specified in academic research. Secondly, we aimed to explore if the ability of each platform is influenced by the level of queries complexity.

Methods

We asked the two platforms to list five studies about each of the following three topics: (1) use of virtual reality in anatomy education, (2) use of stereoscopic virtual reality in anatomy education, (3) use of stereoscopic virtual reality in anatomy education, involving user's interaction with the virtual environment. We assessed if the retrieved studies fulfilled the search criteria, and if their summaries were accurate (if they contained true information about all the educational results of the article's abstract).

Results

The ChatGPT's percentages of successful detection were 100%, 60% and 0% respectively for the three queries. The percentages of accurate summaries were 60%, 20% and 0% respectively. ScholarGPT performed better, with a percentage of successful detection 100%, 60% and 40% respectively. The percentages of accurate summaries were 80%, 60% and 40% respectively. Both platforms showed bias in favor of the educational intervention.

Conclusions

ChatGPT and ScholarGPT are not currently at an adequate level to essentially aid researchers to detect and summarize studies of the anatomy education literature. Ongoing research may increase the ability of those platforms to provide more reliable information.

查看原文本刊更多论文

人工智能能否准确检测和总结解剖学教育文献？ChatGPT与ScholarGPT的比较分析

目的人工智能平台被认为是促进解剖学教师工作和学生学习过程的工具。我们的目的是考察ChatGPT与学术研究中指定的ChatGPT版本ScholarGPT相比，检测和总结解剖学教育文献研究的能力。其次，我们的目标是探索每个平台的能力是否受到查询复杂性水平的影响。方法我们要求两个平台分别列出以下三个主题的五项研究：(1)在解剖学教育中使用虚拟现实；(2)在解剖学教育中使用立体虚拟现实；(3)在解剖学教育中使用立体虚拟现实，涉及用户与虚拟环境的交互。我们评估检索到的研究是否符合检索标准，以及它们的摘要是否准确（是否包含文章摘要中所有教育结果的真实信息）。结果ChatGPT对三个查询的检测成功率分别为100%、60%和0%。总结准确率分别为60%、20%和0%。ScholarGPT表现较好，检测成功率分别为100%、60%和40%。总结准确率分别为80%、60%和40%。两个平台都显示出支持教育干预的偏见。结论schatgpt和ScholarGPT目前的水平不足以从本质上帮助研究者发现和总结解剖学教育文献的研究。正在进行的研究可能会提高这些平台提供更可靠信息的能力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Morphologie Medicine-Anatomy

CiteScore

2.30

自引率

0.00%

发文量

150

审稿时长

25 days

期刊介绍： Morphologie est une revue universitaire avec une ouverture médicale qui sa adresse aux enseignants, aux étudiants, aux chercheurs et aux cliniciens en anatomie et en morphologie. Vous y trouverez les développements les plus actuels de votre spécialité, en France comme a international. Le objectif de Morphologie est d?offrir des lectures privilégiées sous forme de revues générales, d?articles originaux, de mises au point didactiques et de revues de la littérature, qui permettront notamment aux enseignants de optimiser leurs cours et aux spécialistes d?enrichir leurs connaissances.