探索嵌入式 ChatGPT-4 和 ChatGPT-4o 生成 BI-RADS 评分的准确性：放射临床支持中的试点研究。

IF 1.8 4区医学 Q3 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING

Clinical Imaging Pub Date : 2024-10-30 DOI:10.1016/j.clinimag.2024.110335

Dan Nguyen , Arya Rao , Aneesh Mazumder , Marc D. Succi

{"title":"探索嵌入式 ChatGPT-4 和 ChatGPT-4o 生成 BI-RADS 评分的准确性：放射临床支持中的试点研究。","authors":"Dan Nguyen , Arya Rao , Aneesh Mazumder , Marc D. Succi","doi":"10.1016/j.clinimag.2024.110335","DOIUrl":null,"url":null,"abstract":"<div><div>This study evaluates the accuracy of ChatGPT-4 and ChatGPT-4o in generating Breast Imaging Reporting and Data System (BI-RADS) scores from radiographic images. We tested both models using 77 breast cancer images from <span><span>radiopaedia.org</span><svg><path></path></svg></span>, including mammograms and ultrasounds. Images were analyzed in separate sessions to avoid bias. ChatGPT-4 and ChatGPT-4o achieved a 66.2 % accuracy across all BI-RADS cases. Performance was highest in BI-RADS 5 cases, with ChatGPT-4 and ChatGPT4o scoring 84.4 % and 88.9 %, respectively. However, both models struggled with BIRADS 1–3 cases, often assigning higher severity ratings. This study highlights the limitations of current LLMs in accurately grading these images and emphasizes the need for further research in these technologies before clinical integration.</div></div>","PeriodicalId":50680,"journal":{"name":"Clinical Imaging","volume":"117 ","pages":"Article 110335"},"PeriodicalIF":1.8000,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Exploring the accuracy of embedded ChatGPT-4 and ChatGPT-4o in generating BI-RADS scores: a pilot study in radiologic clinical support\",\"authors\":\"Dan Nguyen , Arya Rao , Aneesh Mazumder , Marc D. Succi\",\"doi\":\"10.1016/j.clinimag.2024.110335\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>This study evaluates the accuracy of ChatGPT-4 and ChatGPT-4o in generating Breast Imaging Reporting and Data System (BI-RADS) scores from radiographic images. We tested both models using 77 breast cancer images from <span><span>radiopaedia.org</span><svg><path></path></svg></span>, including mammograms and ultrasounds. Images were analyzed in separate sessions to avoid bias. ChatGPT-4 and ChatGPT-4o achieved a 66.2 % accuracy across all BI-RADS cases. Performance was highest in BI-RADS 5 cases, with ChatGPT-4 and ChatGPT4o scoring 84.4 % and 88.9 %, respectively. However, both models struggled with BIRADS 1–3 cases, often assigning higher severity ratings. This study highlights the limitations of current LLMs in accurately grading these images and emphasizes the need for further research in these technologies before clinical integration.</div></div>\",\"PeriodicalId\":50680,\"journal\":{\"name\":\"Clinical Imaging\",\"volume\":\"117 \",\"pages\":\"Article 110335\"},\"PeriodicalIF\":1.8000,\"publicationDate\":\"2024-10-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Clinical Imaging\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0899707124002651\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Clinical Imaging","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0899707124002651","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

摘要

本研究评估了 ChatGPT-4 和 ChatGPT-4o 从放射影像生成乳腺影像报告和数据系统 (BI-RADS) 评分的准确性。我们使用来自 radiopaedia.org 的 77 幅乳腺癌图像（包括乳房 X 线照相术和超声波检查）对这两种模型进行了测试。为了避免偏差，我们对图像进行了单独分析。ChatGPT-4 和 ChatGPT-4o 在所有 BI-RADS 病例中都达到了 66.2% 的准确率。在 BI-RADS 5 病例中，ChatGPT-4 和 ChatGPT4o 的准确率最高，分别达到 84.4% 和 88.9%。然而，这两种模型在处理 BIRADS 1-3 病例时都很吃力，往往会给出较高的严重程度评级。这项研究强调了目前的 LLM 在对这些图像进行准确分级方面的局限性，并强调了在临床整合之前对这些技术进行进一步研究的必要性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Exploring the accuracy of embedded ChatGPT-4 and ChatGPT-4o in generating BI-RADS scores: a pilot study in radiologic clinical support

This study evaluates the accuracy of ChatGPT-4 and ChatGPT-4o in generating Breast Imaging Reporting and Data System (BI-RADS) scores from radiographic images. We tested both models using 77 breast cancer images from radiopaedia.org, including mammograms and ultrasounds. Images were analyzed in separate sessions to avoid bias. ChatGPT-4 and ChatGPT-4o achieved a 66.2 % accuracy across all BI-RADS cases. Performance was highest in BI-RADS 5 cases, with ChatGPT-4 and ChatGPT4o scoring 84.4 % and 88.9 %, respectively. However, both models struggled with BIRADS 1–3 cases, often assigning higher severity ratings. This study highlights the limitations of current LLMs in accurately grading these images and emphasizes the need for further research in these technologies before clinical integration.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Clinical Imaging 医学-核医学

CiteScore

4.60

自引率

0.00%

发文量

265

审稿时长

35 days

期刊介绍： The mission of Clinical Imaging is to publish, in a timely manner, the very best radiology research from the United States and around the world with special attention to the impact of medical imaging on patient care. The journal''s publications cover all imaging modalities, radiology issues related to patients, policy and practice improvements, and clinically-oriented imaging physics and informatics. The journal is a valuable resource for practicing radiologists, radiologists-in-training and other clinicians with an interest in imaging. Papers are carefully peer-reviewed and selected by our experienced subject editors who are leading experts spanning the range of imaging sub-specialties, which include: -Body Imaging- Breast Imaging- Cardiothoracic Imaging- Imaging Physics and Informatics- Molecular Imaging and Nuclear Medicine- Musculoskeletal and Emergency Imaging- Neuroradiology- Practice, Policy & Education- Pediatric Imaging- Vascular and Interventional Radiology