The effectiveness of ChatGPT in pediatric simulation-based tests of nursing courses in Taiwan: A descriptive study

IF 3.4 3区医学 Q1 NURSING

Clinical Simulation in Nursing Pub Date : 2025-04-24 DOI:10.1016/j.ecns.2025.101732

Hui-Man Huang, Shao-Hui Shu

{"title":"The effectiveness of ChatGPT in pediatric simulation-based tests of nursing courses in Taiwan: A descriptive study","authors":"Hui-Man Huang, Shao-Hui Shu","doi":"10.1016/j.ecns.2025.101732","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>ChatGPT is a generative language model that enhances personalized learning, encourages critical thinking, and supports problem-based learning. However, its use in nursing education requires further validation. This study aimed to assess the performance of ChatGPT models on pediatric simulation-based assessment tests, and compare the test scores with nursing students.</div></div><div><h3>Methods</h3><div>A descriptive study was conducted. The ChatGPT-3.5 and 4.0 were used to complete pediatric simulation-based assessment tests, analyze the content of the responses, and compare scores to those of nursing students. The test included four pediatric simulated scenarios and consisted of 40 items. A passing grade was an average score of 60 or above. A total of 267 fourth-year Associate Degree in Nursing (ADN) program students were recruited for this study.</div></div><div><h3>Results</h3><div>The scores of ChatGPTs were significantly different (<em>p</em> < .05). GPT-4.0 outperformed GPR-3.5, and more effectively conveyed and interpreted images. There are four domains to explain the performance of GPTs: Good Assist, Answer inconsistencies between ChatGPTs, Fabricated information, and Inability to provide comprehensive assessments. Compared to students, the scores of all sections of nursing students were higher than GPTs, except the second round GPT-4.0.</div></div><div><h3>Conclusions</h3><div>ChatGPT is a useful tool in nursing education. However, ChatGPT could not pass all the pediatric simulation-based assessment tests. It is cautious when utilizing ChatGPT to detect real clinical situations.</div></div>","PeriodicalId":48753,"journal":{"name":"Clinical Simulation in Nursing","volume":"102 ","pages":"Article 101732"},"PeriodicalIF":3.4000,"publicationDate":"2025-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Clinical Simulation in Nursing","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1876139925000490","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"NURSING","Score":null,"Total":0}

引用次数: 0

Abstract

Background

ChatGPT is a generative language model that enhances personalized learning, encourages critical thinking, and supports problem-based learning. However, its use in nursing education requires further validation. This study aimed to assess the performance of ChatGPT models on pediatric simulation-based assessment tests, and compare the test scores with nursing students.

Methods

A descriptive study was conducted. The ChatGPT-3.5 and 4.0 were used to complete pediatric simulation-based assessment tests, analyze the content of the responses, and compare scores to those of nursing students. The test included four pediatric simulated scenarios and consisted of 40 items. A passing grade was an average score of 60 or above. A total of 267 fourth-year Associate Degree in Nursing (ADN) program students were recruited for this study.

Results

The scores of ChatGPTs were significantly different (p < .05). GPT-4.0 outperformed GPR-3.5, and more effectively conveyed and interpreted images. There are four domains to explain the performance of GPTs: Good Assist, Answer inconsistencies between ChatGPTs, Fabricated information, and Inability to provide comprehensive assessments. Compared to students, the scores of all sections of nursing students were higher than GPTs, except the second round GPT-4.0.

Conclusions

ChatGPT is a useful tool in nursing education. However, ChatGPT could not pass all the pediatric simulation-based assessment tests. It is cautious when utilizing ChatGPT to detect real clinical situations.

查看原文本刊更多论文

ChatGPT在台湾护理课程儿科模拟测试中的有效性：一项描述性研究

chatgpt是一种生成语言模型，可以增强个性化学习，鼓励批判性思维，并支持基于问题的学习。然而，它在护理教育中的应用需要进一步的验证。本研究旨在评估ChatGPT模型在儿科模拟评估测试中的表现，并与护理专业学生的测试成绩进行比较。方法采用描述性研究。使用ChatGPT-3.5和4.0完成基于儿科模拟的评估测试，分析回答的内容，并与护理学生的得分进行比较。该测试包括四个儿科模拟场景，包括40个项目。及格的平均分数是60分或以上。本研究共招募了267名护理学四年级副学士（ADN）学生。结果两组患者ChatGPTs评分差异有统计学意义(p <；. 05)。GPT-4.0优于GPR-3.5，更有效地传达和解释图像。有四个领域可以解释gpt的表现：良好的辅助，chatgpt之间的答案不一致，伪造的信息，以及无法提供全面的评估。与学生比较，除第二轮GPT-4.0外，护生各部分得分均高于gpt。结论atgpt是护理教育的有效工具。然而，ChatGPT无法通过所有基于儿科模拟的评估测试。在使用ChatGPT检测真实临床情况时要谨慎。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Clinical Simulation in Nursing NURSING-

CiteScore

5.50

自引率

15.40%

发文量

107

期刊介绍： Clinical Simulation in Nursing is an international, peer reviewed journal published online monthly. Clinical Simulation in Nursing is the official journal of the International Nursing Association for Clinical Simulation & Learning (INACSL) and reflects its mission to advance the science of healthcare simulation. We will review and accept articles from other health provider disciplines, if they are determined to be of interest to our readership. The journal accepts manuscripts meeting one or more of the following criteria: Research articles and literature reviews (e.g. systematic, scoping, umbrella, integrative, etc.) about simulation Innovative teaching/learning strategies using simulation Articles updating guidelines, regulations, and legislative policies that impact simulation Leadership for simulation Simulation operations Clinical and academic uses of simulation.