Evaluation of AI chatbots for patient education and information on chronic obstructive pulmonary disease

IF 2.6 4区医学 Q2 CARDIAC & CARDIOVASCULAR SYSTEMS

Heart & Lung Pub Date : 2025-09-11 DOI:10.1016/j.hrtlng.2025.09.002

Pınar Merç , Cansu Şahbaz Pirinççi , Emine Cihan

{"title":"Evaluation of AI chatbots for patient education and information on chronic obstructive pulmonary disease","authors":"Pınar Merç , Cansu Şahbaz Pirinççi , Emine Cihan","doi":"10.1016/j.hrtlng.2025.09.002","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Chronic obstructive pulmonary disease (COPD) is a chronic and progressive disease that affects patients' quality of life and functional capacity. With its widespread use and ease of access, AI chatbots stand out as an alternative source of patient-centered information and education.</div></div><div><h3>Objectives</h3><div>To evaluate the readability and accuracy of information provided by ChatGPT, Gemini, and DeepSeek in COPD.</div></div><div><h3>Methods</h3><div>Ten most frequently asked questions and answers regarding COPD in English were provided using three AI chatbots (ChatGPT-4 Turbo, Gemini 2.0 Flash, DeepSeek R1). Readability was assessed using the Flesch-Kincaid Grade Level (FKGL), while information quality was analyzed by five physiotherapists based on the guidelines. Responses were graded using a 4-point system from “excellent response requiring no explanation” to “unsatisfactory requiring significant explanation.” Statistical analyses were performed on SPSS.</div></div><div><h3>Results</h3><div>Overall, all three AI chatbots responded to questions with similar quality, with Gemini 2.0 providing a statistically higher quality response to question 4 (<em>p</em> < 0.05). In terms of readability of the answers, DeepSeek was found to have better readability on Q5 (12.01), Q8 (9.24), Q9 (13.1) and Q10 (8.73) compared to ChatGPT (Q5:13.9, Q8:11.92, Q9:17.15, Q10:9.88) and Gemini (Q5:18.22, Q8:15.47, Q9:17.42, Q10:9.38). Gemini was observed to produce more complex and academic level answers on more questions (Q4, Q5, Q8).</div></div><div><h3>Conclusions</h3><div>ChatGPT, Gemini, and DeepSeek provided evidence-based answers to frequently asked patient questions about COPD. DeepSeek showed better readability performance for many questions. AI chatbots may serve as a valuable clinical tool for COPD patient education and disease management in the future.</div></div>","PeriodicalId":55064,"journal":{"name":"Heart & Lung","volume":"75 ","pages":"Pages 21-25"},"PeriodicalIF":2.6000,"publicationDate":"2025-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Heart & Lung","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0147956325001876","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CARDIAC & CARDIOVASCULAR SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Chronic obstructive pulmonary disease (COPD) is a chronic and progressive disease that affects patients' quality of life and functional capacity. With its widespread use and ease of access, AI chatbots stand out as an alternative source of patient-centered information and education.

Objectives

To evaluate the readability and accuracy of information provided by ChatGPT, Gemini, and DeepSeek in COPD.

Methods

Ten most frequently asked questions and answers regarding COPD in English were provided using three AI chatbots (ChatGPT-4 Turbo, Gemini 2.0 Flash, DeepSeek R1). Readability was assessed using the Flesch-Kincaid Grade Level (FKGL), while information quality was analyzed by five physiotherapists based on the guidelines. Responses were graded using a 4-point system from “excellent response requiring no explanation” to “unsatisfactory requiring significant explanation.” Statistical analyses were performed on SPSS.

Results

Overall, all three AI chatbots responded to questions with similar quality, with Gemini 2.0 providing a statistically higher quality response to question 4 (p < 0.05). In terms of readability of the answers, DeepSeek was found to have better readability on Q5 (12.01), Q8 (9.24), Q9 (13.1) and Q10 (8.73) compared to ChatGPT (Q5:13.9, Q8:11.92, Q9:17.15, Q10:9.88) and Gemini (Q5:18.22, Q8:15.47, Q9:17.42, Q10:9.38). Gemini was observed to produce more complex and academic level answers on more questions (Q4, Q5, Q8).

Conclusions

ChatGPT, Gemini, and DeepSeek provided evidence-based answers to frequently asked patient questions about COPD. DeepSeek showed better readability performance for many questions. AI chatbots may serve as a valuable clinical tool for COPD patient education and disease management in the future.

查看原文本刊更多论文

人工智能聊天机器人对慢性阻塞性肺疾病患者教育和信息的评估

慢性阻塞性肺疾病（COPD）是一种慢性进行性疾病，影响患者的生活质量和功能能力。由于人工智能聊天机器人的广泛使用和易于访问，它作为以患者为中心的信息和教育的另一种来源脱颖而出。目的评价ChatGPT、Gemini和DeepSeek提供的COPD信息的可读性和准确性。方法使用3个AI聊天机器人（ChatGPT-4 Turbo、Gemini 2.0 Flash、DeepSeek R1）提供与COPD相关的英语最常见问题和答案。可读性采用Flesch-Kincaid Grade Level （FKGL）进行评估，而信息质量由5名物理治疗师根据指南进行分析。回答使用4分制进行评分，从“优秀回答无需解释”到“不满意需要重要解释”。采用SPSS软件进行统计分析。结果总体而言，所有三个AI聊天机器人对问题的回答质量相似，Gemini 2.0对问题4的回答质量更高（p < 0.05）。在答案的可读性方面，DeepSeek在Q5（12.01）、Q8（9.24）、Q9（13.1）和Q10（8.73）上的可读性优于ChatGPT （Q5:13.9、Q8:11.92、Q9:17.15、Q10:9.88）和Gemini （Q5:18.22、Q8:15.47、Q9:17.42、Q10:9.38）。在更多的问题（Q4, Q5, Q8）上，双子座的回答更复杂，更有学术水平。结论：schatgpt、Gemini和DeepSeek为患者关于COPD的常见问题提供了循证答案。DeepSeek在许多问题上表现出更好的可读性。人工智能聊天机器人可能在未来成为COPD患者教育和疾病管理的宝贵临床工具。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Heart & Lung 医学-呼吸系统

CiteScore

4.60

自引率

3.60%

发文量

184

审稿时长

35 days

期刊介绍： Heart & Lung: The Journal of Cardiopulmonary and Acute Care, the official publication of The American Association of Heart Failure Nurses, presents original, peer-reviewed articles on techniques, advances, investigations, and observations related to the care of patients with acute and critical illness and patients with chronic cardiac or pulmonary disorders. The Journal''s acute care articles focus on the care of hospitalized patients, including those in the critical and acute care settings. Because most patients who are hospitalized in acute and critical care settings have chronic conditions, we are also interested in the chronically critically ill, the care of patients with chronic cardiopulmonary disorders, their rehabilitation, and disease prevention. The Journal''s heart failure articles focus on all aspects of the care of patients with this condition. Manuscripts that are relevant to populations across the human lifespan are welcome.