中医执业资格考试GPT与ERNIE:文化背景重要吗?

IF 1.3 4区 医学 Q3 INTEGRATIVE & COMPLEMENTARY MEDICINE
Erfan Ghanad, Christel Weiß, Hui Gao, Christoph Reißfelder, Kamal Hummedah, Lei Han, Leihui Tong, Chengpeng Li, Cui Yang
{"title":"中医执业资格考试GPT与ERNIE:文化背景重要吗?","authors":"Erfan Ghanad, Christel Weiß, Hui Gao, Christoph Reißfelder, Kamal Hummedah, Lei Han, Leihui Tong, Chengpeng Li, Cui Yang","doi":"10.1089/jicm.2024.0902","DOIUrl":null,"url":null,"abstract":"<p><p><b><i>Purpose:</i></b> This study evaluates the performance of large language models (LLMs) in the context of the Chinese National Traditional Chinese Medicine Licensing Examination (TCMLE). <b><i>Materials and Methods:</i></b> We compared the performances of different versions of Generative Pre-trained Transformer (GPT) and Enhanced Representation through Knowledge Integration (ERNIE) using historical TCMLE questions. <b><i>Results:</i></b> ERNIE-4.0 outperformed all other models with an accuracy of 81.7%, followed by ERNIE-3.5 (75.2%), GPT-4o (74.8%), and GPT-4 turbo (50.7%). For questions related to Western internal medicine, all models showed high accuracy above 86.7%. <b><i>Conclusion:</i></b> The study highlights the significance of cultural context in training data, influencing the performance of LLMs in specific medical examinations.</p>","PeriodicalId":29734,"journal":{"name":"Journal of Integrative and Complementary Medicine","volume":" ","pages":""},"PeriodicalIF":1.3000,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?\",\"authors\":\"Erfan Ghanad, Christel Weiß, Hui Gao, Christoph Reißfelder, Kamal Hummedah, Lei Han, Leihui Tong, Chengpeng Li, Cui Yang\",\"doi\":\"10.1089/jicm.2024.0902\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p><b><i>Purpose:</i></b> This study evaluates the performance of large language models (LLMs) in the context of the Chinese National Traditional Chinese Medicine Licensing Examination (TCMLE). <b><i>Materials and Methods:</i></b> We compared the performances of different versions of Generative Pre-trained Transformer (GPT) and Enhanced Representation through Knowledge Integration (ERNIE) using historical TCMLE questions. <b><i>Results:</i></b> ERNIE-4.0 outperformed all other models with an accuracy of 81.7%, followed by ERNIE-3.5 (75.2%), GPT-4o (74.8%), and GPT-4 turbo (50.7%). For questions related to Western internal medicine, all models showed high accuracy above 86.7%. <b><i>Conclusion:</i></b> The study highlights the significance of cultural context in training data, influencing the performance of LLMs in specific medical examinations.</p>\",\"PeriodicalId\":29734,\"journal\":{\"name\":\"Journal of Integrative and Complementary Medicine\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":1.3000,\"publicationDate\":\"2025-07-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Integrative and Complementary Medicine\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1089/jicm.2024.0902\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"INTEGRATIVE & COMPLEMENTARY MEDICINE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Integrative and Complementary Medicine","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1089/jicm.2024.0902","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"INTEGRATIVE & COMPLEMENTARY MEDICINE","Score":null,"Total":0}
引用次数: 0

摘要

目的:本研究评估大语言模型(llm)在中国国家中医药执业资格考试(TCMLE)背景下的表现。材料和方法:我们比较了不同版本的生成预训练转换器(GPT)和通过知识集成增强表示(ERNIE)的性能,使用历史TCMLE问题。结果:ERNIE-4.0以81.7%的准确率优于其他模型,其次是ERNIE-3.5(75.2%)、gpt - 40(74.8%)和GPT-4 turbo(50.7%)。对于与西医相关的问题,所有模型的准确率都在86.7%以上。结论:本研究突出了文化背景在训练数据中的重要性,影响法学硕士在特定医学检查中的表现。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?

Purpose: This study evaluates the performance of large language models (LLMs) in the context of the Chinese National Traditional Chinese Medicine Licensing Examination (TCMLE). Materials and Methods: We compared the performances of different versions of Generative Pre-trained Transformer (GPT) and Enhanced Representation through Knowledge Integration (ERNIE) using historical TCMLE questions. Results: ERNIE-4.0 outperformed all other models with an accuracy of 81.7%, followed by ERNIE-3.5 (75.2%), GPT-4o (74.8%), and GPT-4 turbo (50.7%). For questions related to Western internal medicine, all models showed high accuracy above 86.7%. Conclusion: The study highlights the significance of cultural context in training data, influencing the performance of LLMs in specific medical examinations.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
4.30
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信