大型语言模型在旅行医学认证多项选择题上的表现。

Q3 Medicine
Angelo D'Ambrosio, Francesco Baglivo, Luigi De Angelis, Federico Tecchio, Caterina Rizzo
{"title":"大型语言模型在旅行医学认证多项选择题上的表现。","authors":"Angelo D'Ambrosio, Francesco Baglivo, Luigi De Angelis, Federico Tecchio, Caterina Rizzo","doi":"10.1701/4573.45796","DOIUrl":null,"url":null,"abstract":"<p><p>We benchmarked 40 LLMs on a 40 item travel medicine quiz. Bayesian modelling was used to evaluate accuracy, consistency, parsability, and cost metrics. Accuracy spanned 27.9-97.5%; reasoning tuned frontier models (OpenAI o3, Perplexity Sonar Reasoning) topped the benchmark, whereas local small underperformed. Cost accuracy curves revealed five Pareto optimal systems, with o3 being the current best. These findings confirm the performance of current LLMs as public health knowledge support systems.</p>","PeriodicalId":20887,"journal":{"name":"Recenti progressi in medicina","volume":"116 10","pages":"603-604"},"PeriodicalIF":0.0000,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Performance di large language models su quesiti a risposta multipla per la certificazione in medicina dei viaggi.\",\"authors\":\"Angelo D'Ambrosio, Francesco Baglivo, Luigi De Angelis, Federico Tecchio, Caterina Rizzo\",\"doi\":\"10.1701/4573.45796\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>We benchmarked 40 LLMs on a 40 item travel medicine quiz. Bayesian modelling was used to evaluate accuracy, consistency, parsability, and cost metrics. Accuracy spanned 27.9-97.5%; reasoning tuned frontier models (OpenAI o3, Perplexity Sonar Reasoning) topped the benchmark, whereas local small underperformed. Cost accuracy curves revealed five Pareto optimal systems, with o3 being the current best. These findings confirm the performance of current LLMs as public health knowledge support systems.</p>\",\"PeriodicalId\":20887,\"journal\":{\"name\":\"Recenti progressi in medicina\",\"volume\":\"116 10\",\"pages\":\"603-604\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Recenti progressi in medicina\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1701/4573.45796\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"Medicine\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Recenti progressi in medicina","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1701/4573.45796","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Medicine","Score":null,"Total":0}
引用次数: 0

摘要

我们对40名法学硕士进行了40项旅行医学测试。贝叶斯建模用于评估准确性、一致性、可解析性和成本指标。精度范围为27.9-97.5%;推理调整的前沿模型(OpenAI o3, Perplexity Sonar reasoning)在基准测试中名列前茅,而局部小模型表现不佳。成本精度曲线显示了5个Pareto最优系统,其中o3为当前最佳系统。这些发现证实了当前法学硕士作为公共卫生知识支持系统的表现。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Performance di large language models su quesiti a risposta multipla per la certificazione in medicina dei viaggi.

We benchmarked 40 LLMs on a 40 item travel medicine quiz. Bayesian modelling was used to evaluate accuracy, consistency, parsability, and cost metrics. Accuracy spanned 27.9-97.5%; reasoning tuned frontier models (OpenAI o3, Perplexity Sonar Reasoning) topped the benchmark, whereas local small underperformed. Cost accuracy curves revealed five Pareto optimal systems, with o3 being the current best. These findings confirm the performance of current LLMs as public health knowledge support systems.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Recenti progressi in medicina
Recenti progressi in medicina Medicine-Medicine (all)
CiteScore
0.90
自引率
0.00%
发文量
143
期刊介绍: Giunta ormai al sessantesimo anno, Recenti Progressi in Medicina continua a costituire un sicuro punto di riferimento ed uno strumento di lavoro fondamentale per l"ampliamento dell"orizzonte culturale del medico italiano. Recenti Progressi in Medicina è una rivista di medicina interna. Ciò significa il recupero di un"ottica globale e integrata, idonea ad evitare sia i particolarismi della informazione specialistica sia la frammentazione di quella generalista.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信