Performance of GPT-4 on Chinese Nursing Examination: Potentials for AI-Assisted Nursing Education Using Large Language Models.

IF 4.6 Q2 MATERIALS SCIENCE, BIOMATERIALS

ACS Applied Bio Materials Pub Date : 2024-11-01 Epub Date: 2024-07-05 DOI:10.1097/NNE.0000000000001679

Yiqun Miao, Yuan Luo, Yuhan Zhao, Jiawei Li, Mingxuan Liu, Huiying Wang, Yuling Chen, Ying Wu

{"title":"Performance of GPT-4 on Chinese Nursing Examination: Potentials for AI-Assisted Nursing Education Using Large Language Models.","authors":"Yiqun Miao, Yuan Luo, Yuhan Zhao, Jiawei Li, Mingxuan Liu, Huiying Wang, Yuling Chen, Ying Wu","doi":"10.1097/NNE.0000000000001679","DOIUrl":null,"url":null,"abstract":"Background: The performance of GPT-4 in nursing examinations within the Chinese context has not yet been thoroughly evaluated.Objective: To assess the performance of GPT-4 on multiple-choice and open-ended questions derived from nursing examinations in the Chinese context.Methods: The data sets of the Chinese National Nursing Licensure Examination spanning 2021 to 2023 were used to evaluate the accuracy of GPT-4 in multiple-choice questions. The performance of GPT-4 on open-ended questions was examined using 18 case-based questions.Results: For multiple-choice questions, GPT-4 achieved an accuracy of 71.0% (511/720). For open-ended questions, the responses were evaluated for cosine similarity, logical consistency, and information quality, all of which were found to be at a moderate level.Conclusion: GPT-4 performed well at addressing queries on basic knowledge. However, it has notable limitations in answering open-ended questions. Nursing educators should weigh the benefits and challenges of GPT-4 for integration into nursing education.","PeriodicalId":2,"journal":{"name":"ACS Applied Bio Materials","volume":null,"pages":null},"PeriodicalIF":4.6000,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACS Applied Bio Materials","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/NNE.0000000000001679","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/7/5 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"MATERIALS SCIENCE, BIOMATERIALS","Score":null,"Total":0}

引用次数: 0

Abstract

Background: The performance of GPT-4 in nursing examinations within the Chinese context has not yet been thoroughly evaluated.

Objective: To assess the performance of GPT-4 on multiple-choice and open-ended questions derived from nursing examinations in the Chinese context.

Methods: The data sets of the Chinese National Nursing Licensure Examination spanning 2021 to 2023 were used to evaluate the accuracy of GPT-4 in multiple-choice questions. The performance of GPT-4 on open-ended questions was examined using 18 case-based questions.

Results: For multiple-choice questions, GPT-4 achieved an accuracy of 71.0% (511/720). For open-ended questions, the responses were evaluated for cosine similarity, logical consistency, and information quality, all of which were found to be at a moderate level.

Conclusion: GPT-4 performed well at addressing queries on basic knowledge. However, it has notable limitations in answering open-ended questions. Nursing educators should weigh the benefits and challenges of GPT-4 for integration into nursing education.

查看原文本刊更多论文

GPT-4 在中国护理考试中的表现：使用大语言模型进行人工智能辅助护理教育的潜力。

背景GPT-4 在中国护理考试中的表现尚未得到全面评估：评估 GPT-4 在中国护理考试选择题和开放性试题中的表现：方法：使用 2021 年至 2023 年中国国家护士执业资格考试的数据集来评估 GPT-4 在多项选择题中的准确性。结果：在多项选择题中，GPT-4的准确率为100%；在开放式问题中，GPT-4的准确率为100%；在18道案例题中，GPT-4的准确率为100%：在多项选择题中，GPT-4 的准确率为 71.0%（511/720）。对于开放式问题，对回答的余弦相似性、逻辑一致性和信息质量进行了评估，结果发现所有这些都处于中等水平：结论：GPT-4 在解决基础知识查询方面表现良好。结论：GPT-4 在回答基础知识询问方面表现良好，但在回答开放式问题方面有明显的局限性。护理教育者应权衡 GPT-4 的优势和挑战，以便将其纳入护理教育中。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

ACS Applied Bio Materials Chemistry-Chemistry (all)

CiteScore

9.40

自引率

2.10%

发文量

464