Can artificial intelligence pass the written European Board of Hand Surgery exam?

IF 1 4区医学 Q4 ORTHOPEDICS

Hand Surgery & Rehabilitation Pub Date : 2025-09-01 DOI:10.1016/j.hansur.2025.102197

Sinan Mert , Lindsay Muir , Benedikt Fuchs , Vanessa Lucksch , Felix H. Vollbach , Elisabeth M. Haas-Lützenberger , Riccardo E. Giunta , Nikolaus Thierfelder , Wolfram Demmer

{"title":"Can artificial intelligence pass the written European Board of Hand Surgery exam?","authors":"Sinan Mert , Lindsay Muir , Benedikt Fuchs , Vanessa Lucksch , Felix H. Vollbach , Elisabeth M. Haas-Lützenberger , Riccardo E. Giunta , Nikolaus Thierfelder , Wolfram Demmer","doi":"10.1016/j.hansur.2025.102197","DOIUrl":null,"url":null,"abstract":"<div><div>Various artificial intelligence-based applications have emerged as transformative tools across numerous domains. Among these, ChatGPT has earned global recognition with its capacity for dynamic user interaction and holds significant potential in the medical sector. However, the subject-specific accuracy of ChatGPT remains a matter of debate.</div><div>This study assesses the capabilities and knowledge of different artificial intelligence chatbots (ChatGPT, Google Gemini, and Claude) in the domain of hand surgery. Each chatbot conducted a full written EBHS exam. The test results were analyzed according to the EBHS-guidelines, focused on the total scores and the ratio of correct to incorrect responses for each artificial intelligence model. Findings revealed that three out of the four chatbots achieved passing scores on the exam. Notably, ChatGPT-4o1 demonstrated significantly superior performance.</div><div>This study highlights the subject-specific expertise of different artificial intelligence programs within the specialized field of hand surgery while also underscoring their variability and limitations.</div></div>","PeriodicalId":54301,"journal":{"name":"Hand Surgery & Rehabilitation","volume":"44 4","pages":"Article 102197"},"PeriodicalIF":1.0000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Hand Surgery & Rehabilitation","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2468122925001197","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"ORTHOPEDICS","Score":null,"Total":0}

引用次数: 0

Abstract

Various artificial intelligence-based applications have emerged as transformative tools across numerous domains. Among these, ChatGPT has earned global recognition with its capacity for dynamic user interaction and holds significant potential in the medical sector. However, the subject-specific accuracy of ChatGPT remains a matter of debate.

This study assesses the capabilities and knowledge of different artificial intelligence chatbots (ChatGPT, Google Gemini, and Claude) in the domain of hand surgery. Each chatbot conducted a full written EBHS exam. The test results were analyzed according to the EBHS-guidelines, focused on the total scores and the ratio of correct to incorrect responses for each artificial intelligence model. Findings revealed that three out of the four chatbots achieved passing scores on the exam. Notably, ChatGPT-4o1 demonstrated significantly superior performance.

This study highlights the subject-specific expertise of different artificial intelligence programs within the specialized field of hand surgery while also underscoring their variability and limitations.

查看原文本刊更多论文

人工智能能通过欧洲手外科委员会笔试吗？

各种基于人工智能的应用程序已经成为许多领域的变革性工具。其中，ChatGPT以其动态用户交互能力赢得了全球的认可，在医疗领域具有巨大的潜力。然而，ChatGPT的特定主题的准确性仍然是一个有争议的问题。本研究评估了不同人工智能聊天机器人（ChatGPT、谷歌Gemini和Claude）在手部手术领域的能力和知识。每个聊天机器人都进行了一次完整的EBHS书面考试。测试结果根据ebhs指南进行分析，重点关注每个人工智能模型的总分和正确与错误回答的比例。调查结果显示，四个聊天机器人中有三个在考试中取得了及格分数。值得注意的是，chatgpt - 401表现出明显优越的性能。本研究强调了不同人工智能程序在手外科专业领域的专业知识，同时也强调了它们的可变性和局限性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Hand Surgery & Rehabilitation Medicine-Surgery

CiteScore

1.70

自引率

27.30%

发文量

审稿时长

49 days

期刊介绍： As the official publication of the French, Belgian and Swiss Societies for Surgery of the Hand, as well as of the French Society of Rehabilitation of the Hand & Upper Limb, ''Hand Surgery and Rehabilitation'' - formerly named "Chirurgie de la Main" - publishes original articles, literature reviews, technical notes, and clinical cases. It is indexed in the main international databases (including Medline). Initially a platform for French-speaking hand surgeons, the journal will now publish its articles in English to disseminate its author''s scientific findings more widely. The journal also includes a biannual supplement in French, the monograph of the French Society for Surgery of the Hand, where comprehensive reviews in the fields of hand, peripheral nerve and upper limb surgery are presented. Organe officiel de la Société française de chirurgie de la main, de la Société française de Rééducation de la main (SFRM-GEMMSOR), de la Société suisse de chirurgie de la main et du Belgian Hand Group, indexée dans les grandes bases de données internationales (Medline, Embase, Pascal, Scopus), Hand Surgery and Rehabilitation - anciennement titrée Chirurgie de la main - publie des articles originaux, des revues de la littérature, des notes techniques, des cas clinique. Initialement plateforme d''expression francophone de la spécialité, la revue s''oriente désormais vers l''anglais pour devenir une référence scientifique et de formation de la spécialité en France et en Europe. Avec 6 publications en anglais par an, la revue comprend également un supplément biannuel, la monographie du GEM, où sont présentées en français, des mises au point complètes dans les domaines de la chirurgie de la main, des nerfs périphériques et du membre supérieur.