The Quality of Information Produced by ChatGPT About Conditions Managed by Interventional Radiologists

IF 1.4 4区医学 Q2 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING

Journal of Medical Imaging and Radiation Oncology Pub Date : 2025-08-09 DOI:10.1111/1754-9485.13881

Ruairidh Read, Matthew Lukies

{"title":"The Quality of Information Produced by ChatGPT About Conditions Managed by Interventional Radiologists","authors":"Ruairidh Read, Matthew Lukies","doi":"10.1111/1754-9485.13881","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Introduction</h3>\n \n <p>The emergence of search engines powered by artificial intelligence and large language models (LLMs), such as ChatGPT, provides easy access to seemingly accurate health information. However, the accuracy of the information produced is uncertain. The purpose of this research is to assess the quality of information produced by ChatGPT about the treatment of health conditions commonly managed by Interventional Radiologists (IRs).</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>ChatGPT was asked “what is the best treatment” in relation to six conditions commonly managed by IRs. The output statements were assessed using the DISCERN instrument and compared against the current evidence base for the management of those conditions.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Six conditions were assessed. The mean overall score for the ChatGPT output statements was 1.3 compared to 3.8 for the reference articles. This poor performance by ChatGPT is largely attributable to the lack of transparency regarding sources. Although ChatGPT performed well in some areas such as presenting information in an unbiased manner, it showed significant weaknesses regarding source materials, the risks and benefits of each treatment, and the treatment's mechanism of action.</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>LLMs signify a considerable shift in how patients obtain and consume medical information. Understanding the strengths and weaknesses of ChatGPT's outputs regarding conditions commonly treated by IRs will enable tailored messaging and constructive discussions with patients in consultation with their IR.</p>\n </section>\n </div>","PeriodicalId":16218,"journal":{"name":"Journal of Medical Imaging and Radiation Oncology","volume":"69 7","pages":"715-724"},"PeriodicalIF":1.4000,"publicationDate":"2025-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Medical Imaging and Radiation Oncology","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/1754-9485.13881","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction

The emergence of search engines powered by artificial intelligence and large language models (LLMs), such as ChatGPT, provides easy access to seemingly accurate health information. However, the accuracy of the information produced is uncertain. The purpose of this research is to assess the quality of information produced by ChatGPT about the treatment of health conditions commonly managed by Interventional Radiologists (IRs).

Methods

ChatGPT was asked “what is the best treatment” in relation to six conditions commonly managed by IRs. The output statements were assessed using the DISCERN instrument and compared against the current evidence base for the management of those conditions.

Results

Six conditions were assessed. The mean overall score for the ChatGPT output statements was 1.3 compared to 3.8 for the reference articles. This poor performance by ChatGPT is largely attributable to the lack of transparency regarding sources. Although ChatGPT performed well in some areas such as presenting information in an unbiased manner, it showed significant weaknesses regarding source materials, the risks and benefits of each treatment, and the treatment's mechanism of action.

Conclusion

LLMs signify a considerable shift in how patients obtain and consume medical information. Understanding the strengths and weaknesses of ChatGPT's outputs regarding conditions commonly treated by IRs will enable tailored messaging and constructive discussions with patients in consultation with their IR.

Abstract Image

查看原文本刊更多论文

由ChatGPT产生的关于介入放射科医生管理的条件的信息质量。

简介：由人工智能和大型语言模型（llm）驱动的搜索引擎的出现，如ChatGPT，提供了访问看似准确的健康信息的便捷途径。然而，所产生的信息的准确性是不确定的。本研究的目的是评估ChatGPT提供的有关介入放射科医生（IRs）通常管理的健康状况治疗的信息的质量。方法：ChatGPT被问及“什么是最好的治疗”，涉及到IRs通常管理的六种情况。使用DISCERN工具对输出报表进行评估，并与当前管理这些条件的证据基础进行比较。结果：评估了6种情况。ChatGPT输出语句的平均总分是1.3，而参考文章的平均总分是3.8。ChatGPT的这种糟糕表现在很大程度上是由于缺乏关于来源的透明度。尽管ChatGPT在某些方面表现良好，例如以公正的方式呈现信息，但它在来源材料、每种治疗的风险和益处以及治疗的作用机制方面显示出明显的弱点。结论：法学硕士表明患者获取和消费医疗信息的方式发生了相当大的转变。了解ChatGPT关于IRs通常治疗的疾病的输出的优势和劣势，将有助于在与其IR协商时与患者进行有针对性的信息传递和建设性的讨论。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Medical Imaging and Radiation Oncology 医学-核医学

CiteScore

3.30

自引率

6.20%

发文量

133

审稿时长

6-12 weeks

期刊介绍： Journal of Medical Imaging and Radiation Oncology (formerly Australasian Radiology) is the official journal of The Royal Australian and New Zealand College of Radiologists, publishing articles of scientific excellence in radiology and radiation oncology. Manuscripts are judged on the basis of their contribution of original data and ideas or interpretation. All articles are peer reviewed.