RadGPT: A System Based on a Large Language Model That Generates Sets of Patient-Centered Materials to Explain Radiology Report Information

IF 5.1 3区医学 Q1 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING

Journal of the American College of Radiology Pub Date : 2025-09-01 DOI:10.1016/j.jacr.2025.06.013

Sanna E. Herwald MD, PhD , Preya Shah MD, PhD , Andrew Johnston MD, MBA , Cameron Olsen MD , Jean-Benoit Delbrouck PhD , Curtis P. Langlotz MD, PhD

{"title":"RadGPT: A System Based on a Large Language Model That Generates Sets of Patient-Centered Materials to Explain Radiology Report Information","authors":"Sanna E. Herwald MD, PhD , Preya Shah MD, PhD , Andrew Johnston MD, MBA , Cameron Olsen MD , Jean-Benoit Delbrouck PhD , Curtis P. Langlotz MD, PhD","doi":"10.1016/j.jacr.2025.06.013","DOIUrl":null,"url":null,"abstract":"<div><h3>Purpose</h3><div>The 21st Century Cures Act final rule requires that patients have real-time access to their radiology reports, which contain technical language. The objective of this study to was to use a novel system called RadGPT, which integrates concept extraction and a large language model (LLM), to help patients understand their radiology reports.</div></div><div><h3>Methods</h3><div>RadGPT generated 150 concept explanations and 390 question-and-answer pairs from 30 radiology report impressions from between 2012 and 2020. The extracted concepts were used to create concept-based explanations, as well as concept-based question-and-answer pairs for which questions were generated using either a fixed template or an LLM. Additionally, report-based question-and-answer pairs were generated directly from the impression using an LLM without concept extraction. One board-certified radiologist and four radiology residents rated the material quality using a standardized rubric.</div></div><div><h3>Results</h3><div>Concept-based LLM-generated questions were of significantly higher quality than concept-based template-generated questions (<em>P</em> < .001). Excluding those template-based question-and-answer pairs from further analysis, nearly all (>95%) of RadGPT-generated materials were rated highly, with at least 50% receiving the highest possible ranking from all five raters. No answers or explanations were rated as likely to affect the safety or effectiveness of patient care. Report-level LLM-based questions and answers were rated particularly highly, with 92% of report-level LLM-based questions and 61% of the corresponding report-level answers receiving the highest rating from all raters.</div></div><div><h3>Conclusions</h3><div>The educational tool RadGPT generated high-quality explanations and question-and-answer pairs that were personalized for each radiology report, unlikely to produce harmful explanations, and likely to enhance patient understanding of radiology information.</div></div>","PeriodicalId":49044,"journal":{"name":"Journal of the American College of Radiology","volume":"22 9","pages":"Pages 1050-1059"},"PeriodicalIF":5.1000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the American College of Radiology","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1546144025003217","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose

The 21st Century Cures Act final rule requires that patients have real-time access to their radiology reports, which contain technical language. The objective of this study to was to use a novel system called RadGPT, which integrates concept extraction and a large language model (LLM), to help patients understand their radiology reports.

Methods

RadGPT generated 150 concept explanations and 390 question-and-answer pairs from 30 radiology report impressions from between 2012 and 2020. The extracted concepts were used to create concept-based explanations, as well as concept-based question-and-answer pairs for which questions were generated using either a fixed template or an LLM. Additionally, report-based question-and-answer pairs were generated directly from the impression using an LLM without concept extraction. One board-certified radiologist and four radiology residents rated the material quality using a standardized rubric.

Results

Concept-based LLM-generated questions were of significantly higher quality than concept-based template-generated questions (P < .001). Excluding those template-based question-and-answer pairs from further analysis, nearly all (>95%) of RadGPT-generated materials were rated highly, with at least 50% receiving the highest possible ranking from all five raters. No answers or explanations were rated as likely to affect the safety or effectiveness of patient care. Report-level LLM-based questions and answers were rated particularly highly, with 92% of report-level LLM-based questions and 61% of the corresponding report-level answers receiving the highest rating from all raters.

Conclusions

The educational tool RadGPT generated high-quality explanations and question-and-answer pairs that were personalized for each radiology report, unlikely to produce harmful explanations, and likely to enhance patient understanding of radiology information.

查看原文本刊更多论文

RadGPT：基于大型语言模型的系统，生成以患者为中心的材料集来解释放射学报告信息。

目的：治愈法案最终规则要求患者能够实时访问他们的放射报告，其中包含技术语言。我们的目标是使用一种称为RadGPT的新系统，它集成了概念提取和大型语言模型（LLM），以帮助患者理解他们的放射学报告。方法：RadGPT从2012年至2020年的30份放射学报告印象中生成150个概念解释和390对问答。提取的概念用于创建基于概念的解释，以及基于概念的问答对，其中使用固定模板或LLM生成问题。此外，使用法学硕士直接从印象中生成基于报告的问答对，而不需要概念提取。一名委员会认证的放射科医生和4名放射科住院医师使用标准化的标准来评估材料质量。结果：基于概念的llm生成问题的质量显著高于基于概念的模板生成问题（p < 0.001）。排除进一步分析中基于模板的问答对，几乎所有（约95%）的radgpt生成的材料都获得了很高的评价，至少有50%的材料在所有5位评分者中获得了最高的评分。没有答案或解释被评为可能影响患者护理的安全性或有效性。报告级法学硕士问题和答案的评分特别高，92%的报告级法学硕士问题和61%的相应报告级答案获得了所有评分者的最高评分。讨论：教育工具RadGPT生成高质量的解释和问答对，这些解释和问答对每个放射学报告都是个性化的，不太可能产生有害的解释，并可能增强患者对放射学信息的理解。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of the American College of Radiology RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING-

CiteScore

6.30

自引率

8.90%

发文量

312

审稿时长

34 days

期刊介绍： The official journal of the American College of Radiology, JACR informs its readers of timely, pertinent, and important topics affecting the practice of diagnostic radiologists, interventional radiologists, medical physicists, and radiation oncologists. In so doing, JACR improves their practices and helps optimize their role in the health care system. By providing a forum for informative, well-written articles on health policy, clinical practice, practice management, data science, and education, JACR engages readers in a dialogue that ultimately benefits patient care.