Capability of multimodal large language models to interpret pediatric radiological images.

IF 2.1 3区医学 Q2 PEDIATRICS

Pediatric Radiology Pub Date : 2024-09-01 Epub Date: 2024-08-12 DOI:10.1007/s00247-024-06025-0

Thomas P Reith, Donna M D'Alessandro, Michael P D'Alessandro

{"title":"Capability of multimodal large language models to interpret pediatric radiological images.","authors":"Thomas P Reith, Donna M D'Alessandro, Michael P D'Alessandro","doi":"10.1007/s00247-024-06025-0","DOIUrl":null,"url":null,"abstract":"Background: There is a dearth of artificial intelligence (AI) development and research dedicated to pediatric radiology. The newest iterations of large language models (LLMs) like ChatGPT can process image and video input in addition to text. They are thus theoretically capable of providing impressions of input radiological images.Objective: To assess the ability of multimodal LLMs to interpret pediatric radiological images.Materials and methods: Thirty medically significant cases were collected and submitted to GPT-4 (OpenAI, San Francisco, CA), Gemini 1.5 Pro (Google, Mountain View, CA), and Claude 3 Opus (Anthropic, San Francisco, CA) with a short history for a total of 90 images. AI responses were recorded and independently assessed for accuracy by a resident and attending physician. 95% confidence intervals were determined using the adjusted Wald method.Results: Overall, the models correctly diagnosed 27.8% (25/90) of images (95% CI=19.5-37.8%), were partially correct for 13.3% (12/90) of images (95% CI=2.7-26.4%), and were incorrect for 58.9% (53/90) of images (95% CI=48.6-68.5%).Conclusion: Multimodal LLMs are not yet capable of interpreting pediatric radiological images.","PeriodicalId":19755,"journal":{"name":"Pediatric Radiology","volume":" ","pages":"1729-1737"},"PeriodicalIF":2.1000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pediatric Radiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s00247-024-06025-0","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/12 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"PEDIATRICS","Score":null,"Total":0}

引用次数: 0

Abstract

Background: There is a dearth of artificial intelligence (AI) development and research dedicated to pediatric radiology. The newest iterations of large language models (LLMs) like ChatGPT can process image and video input in addition to text. They are thus theoretically capable of providing impressions of input radiological images.

Objective: To assess the ability of multimodal LLMs to interpret pediatric radiological images.

Materials and methods: Thirty medically significant cases were collected and submitted to GPT-4 (OpenAI, San Francisco, CA), Gemini 1.5 Pro (Google, Mountain View, CA), and Claude 3 Opus (Anthropic, San Francisco, CA) with a short history for a total of 90 images. AI responses were recorded and independently assessed for accuracy by a resident and attending physician. 95% confidence intervals were determined using the adjusted Wald method.

Results: Overall, the models correctly diagnosed 27.8% (25/90) of images (95% CI=19.5-37.8%), were partially correct for 13.3% (12/90) of images (95% CI=2.7-26.4%), and were incorrect for 58.9% (53/90) of images (95% CI=48.6-68.5%).

Conclusion: Multimodal LLMs are not yet capable of interpreting pediatric radiological images.

Abstract Image

查看原文本刊更多论文

多模态大语言模型解读儿科放射影像的能力。

背景：专门针对儿科放射学的人工智能（AI）开发和研究十分匮乏。最新迭代的大型语言模型（LLMs），如 ChatGPT，除了能处理文本外，还能处理图像和视频输入。因此，理论上它们能够提供输入的放射图像的印象：评估多模态语言模型解读儿科放射影像的能力：收集并向 GPT-4（OpenAI，加利福尼亚州旧金山）、Gemini 1.5 Pro（谷歌，加利福尼亚州山景城）和 Claude 3 Opus（Anthropic，加利福尼亚州旧金山）提交了 30 个有医学意义的病例以及简短病史，共计 90 幅图像。人工智能反应由一名住院医师和一名主治医师记录并独立评估其准确性。95% 的置信区间采用调整 Wald 法确定：总体而言，模型正确诊断了 27.8% (25/90) 的图像（95% CI=19.5-37.8%），部分正确诊断了 13.3% (12/90) 的图像（95% CI=2.7-26.4%），错误诊断了 58.9% (53/90) 的图像（95% CI=48.6-68.5%）：结论：多模态 LLM 尚不能解读儿科放射影像。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Pediatric Radiology 医学-核医学

CiteScore

4.40

自引率

17.40%

发文量

300

审稿时长

3-6 weeks

期刊介绍： Official Journal of the European Society of Pediatric Radiology, the Society for Pediatric Radiology and the Asian and Oceanic Society for Pediatric Radiology Pediatric Radiology informs its readers of new findings and progress in all areas of pediatric imaging and in related fields. This is achieved by a blend of original papers, complemented by reviews that set out the present state of knowledge in a particular area of the specialty or summarize specific topics in which discussion has led to clear conclusions. Advances in technology, methodology, apparatus and auxiliary equipment are presented, and modifications of standard techniques are described. Manuscripts submitted for publication must contain a statement to the effect that all human studies have been reviewed by the appropriate ethics committee and have therefore been performed in accordance with the ethical standards laid down in an appropriate version of the 1964 Declaration of Helsinki. It should also be stated clearly in the text that all persons gave their informed consent prior to their inclusion in the study. Details that might disclose the identity of the subjects under study should be omitted.