人工智能生成的哮喘患者问题的答案：AIR-Asthma研究。

IF 6.6 1区医学 Q1 ALLERGY

Journal of Allergy and Clinical Immunology-In Practice Pub Date : 2025-09-01 DOI:10.1016/j.jaip.2025.04.051

Mattia Nigro MD , Andrea Aliverti PhD , Alessandra Angelucci PhD , Fulvio Braido MD, PhD , Giorgio W. Canonica MD , Apostolos Bossios MD, PhD , Hilary Pinnock MD , Jeanette Boyd PhD , Pippa Powell PhD , Stefano Aliberti MD , Artificial Intelligence Responses on Asthma Study Task Force

{"title":"人工智能生成的哮喘患者问题的答案：AIR-Asthma研究。","authors":"Mattia Nigro MD , Andrea Aliverti PhD , Alessandra Angelucci PhD , Fulvio Braido MD, PhD , Giorgio W. Canonica MD , Apostolos Bossios MD, PhD , Hilary Pinnock MD , Jeanette Boyd PhD , Pippa Powell PhD , Stefano Aliberti MD , Artificial Intelligence Responses on Asthma Study Task Force","doi":"10.1016/j.jaip.2025.04.051","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Asthma is a prevalent chronic respiratory disease requiring ongoing patient education and individualized management. The increasing reliance on digital tools, particularly generative artificial intelligence (AI), to answer health-related questions has raised concerns about the accuracy, reliability, and comprehensibility of AI-generated information for people living with asthma.</div></div><div><h3>Objective</h3><div>To evaluate systematically the reliability, accuracy, comprehensiveness, and understandability of responses generated by three widely used artificial intelligence–based chatbots (ChatGPT, Bard, and Copilot) to common questions formulated by people with asthma.</div></div><div><h3>Methods</h3><div>In this cross-sectional study, 15 questions regarding asthma management were formulated by patients and categorized by difficulty. Responses from ChatGPT, Bard, and Copilot were evaluated by international experts for accuracy and comprehensiveness, and by patient representatives for understandability. Reliability was assessed through consistency testing across devices. We conducted a blinded evaluation.</div></div><div><h3>Results</h3><div>A total of 21 experts and 16 patient representatives participated in the evaluation. ChatGPT demonstrated the highest reliability (15 of 15 responses), accuracy (median score, 9.0; interquartile range [IQR], 7.0-9.0), and comprehensiveness (median score, 8.0; IQR, 8.0-9.0) compared with Bard and Copilot (<em>P</em> < .0001). Bard achieved superior scores in understandability (median score, 9.0; IQR, 8.0-10.0; <em>P</em> < .0001). Performance differences were consistent across question difficulty levels.</div></div><div><h3>Conclusions</h3><div>Artificial intelligence–driven chatbots can provide generally accurate and understandable responses to asthma-related questions. Variability in reliability and accuracy underscores the need for caution in clinical contexts. Artificial intelligence tools may complement but cannot replace professional medical advice in asthma management.</div></div>","PeriodicalId":51323,"journal":{"name":"Journal of Allergy and Clinical Immunology-In Practice","volume":"13 9","pages":"Pages 2390-2396"},"PeriodicalIF":6.6000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Artificial Intelligence–Generated Answers to Patients’ Questions on Asthma: The Artificial Intelligence Responses on Asthma Study\",\"authors\":\"Mattia Nigro MD , Andrea Aliverti PhD , Alessandra Angelucci PhD , Fulvio Braido MD, PhD , Giorgio W. Canonica MD , Apostolos Bossios MD, PhD , Hilary Pinnock MD , Jeanette Boyd PhD , Pippa Powell PhD , Stefano Aliberti MD , Artificial Intelligence Responses on Asthma Study Task Force\",\"doi\":\"10.1016/j.jaip.2025.04.051\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Background</h3><div>Asthma is a prevalent chronic respiratory disease requiring ongoing patient education and individualized management. The increasing reliance on digital tools, particularly generative artificial intelligence (AI), to answer health-related questions has raised concerns about the accuracy, reliability, and comprehensibility of AI-generated information for people living with asthma.</div></div><div><h3>Objective</h3><div>To evaluate systematically the reliability, accuracy, comprehensiveness, and understandability of responses generated by three widely used artificial intelligence–based chatbots (ChatGPT, Bard, and Copilot) to common questions formulated by people with asthma.</div></div><div><h3>Methods</h3><div>In this cross-sectional study, 15 questions regarding asthma management were formulated by patients and categorized by difficulty. Responses from ChatGPT, Bard, and Copilot were evaluated by international experts for accuracy and comprehensiveness, and by patient representatives for understandability. Reliability was assessed through consistency testing across devices. We conducted a blinded evaluation.</div></div><div><h3>Results</h3><div>A total of 21 experts and 16 patient representatives participated in the evaluation. ChatGPT demonstrated the highest reliability (15 of 15 responses), accuracy (median score, 9.0; interquartile range [IQR], 7.0-9.0), and comprehensiveness (median score, 8.0; IQR, 8.0-9.0) compared with Bard and Copilot (<em>P</em> < .0001). Bard achieved superior scores in understandability (median score, 9.0; IQR, 8.0-10.0; <em>P</em> < .0001). Performance differences were consistent across question difficulty levels.</div></div><div><h3>Conclusions</h3><div>Artificial intelligence–driven chatbots can provide generally accurate and understandable responses to asthma-related questions. Variability in reliability and accuracy underscores the need for caution in clinical contexts. Artificial intelligence tools may complement but cannot replace professional medical advice in asthma management.</div></div>\",\"PeriodicalId\":51323,\"journal\":{\"name\":\"Journal of Allergy and Clinical Immunology-In Practice\",\"volume\":\"13 9\",\"pages\":\"Pages 2390-2396\"},\"PeriodicalIF\":6.6000,\"publicationDate\":\"2025-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Allergy and Clinical Immunology-In Practice\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2213219825004209\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ALLERGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Allergy and Clinical Immunology-In Practice","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2213219825004209","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ALLERGY","Score":null,"Total":0}

引用次数: 0

摘要

背景：哮喘是一种常见的慢性呼吸道疾病，需要持续的患者教育和个体化治疗。越来越多地依赖数字工具，特别是生成式人工智能（AI）来回答与健康相关的问题，这引起了人们对人工智能生成的哮喘患者信息的准确性、可靠性和可理解性的担忧。目的：系统评估三种广泛使用的基于人工智能的聊天机器人（ChatGPT、Bard、Copilot）对哮喘患者提出的常见问题的回答的可靠性、准确性、全面性和可理解性。方法：在横断面研究中，由患者自行制定15个哮喘管理问题，并按难度进行分类。ChatGPT、Bard和Copilot的回答由国际专家评估其准确性和全面性，并由患者代表评估其可理解性。通过跨设备的一致性测试来评估可靠性。进行盲法评价。结果：共有21名专家和16名患者代表参与了评估。与Bard和Copilot相比，ChatGPT表现出最高的信度（15/15个反应）、准确性（中位数得分9.0 [IQR 7.0-9.0]）和综合性（8.0 [8.0-9.0]）（P < 0.0001）。巴德在可理解性方面得分更高（中位得分9.0 [8.0-10.0]）（P < 0.0001）。不同难度的问题表现差异是一致的。结论：ai驱动的聊天机器人可以对哮喘相关问题提供大致准确且可理解的回答。可靠性和准确性的可变性强调了在临床环境中谨慎的必要性。人工智能工具可以补充但不能取代哮喘管理方面的专业医疗建议。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Artificial Intelligence–Generated Answers to Patients’ Questions on Asthma: The Artificial Intelligence Responses on Asthma Study

Background

Asthma is a prevalent chronic respiratory disease requiring ongoing patient education and individualized management. The increasing reliance on digital tools, particularly generative artificial intelligence (AI), to answer health-related questions has raised concerns about the accuracy, reliability, and comprehensibility of AI-generated information for people living with asthma.

Objective

To evaluate systematically the reliability, accuracy, comprehensiveness, and understandability of responses generated by three widely used artificial intelligence–based chatbots (ChatGPT, Bard, and Copilot) to common questions formulated by people with asthma.

Methods

In this cross-sectional study, 15 questions regarding asthma management were formulated by patients and categorized by difficulty. Responses from ChatGPT, Bard, and Copilot were evaluated by international experts for accuracy and comprehensiveness, and by patient representatives for understandability. Reliability was assessed through consistency testing across devices. We conducted a blinded evaluation.

Results

A total of 21 experts and 16 patient representatives participated in the evaluation. ChatGPT demonstrated the highest reliability (15 of 15 responses), accuracy (median score, 9.0; interquartile range [IQR], 7.0-9.0), and comprehensiveness (median score, 8.0; IQR, 8.0-9.0) compared with Bard and Copilot (P < .0001). Bard achieved superior scores in understandability (median score, 9.0; IQR, 8.0-10.0; P < .0001). Performance differences were consistent across question difficulty levels.

Conclusions

Artificial intelligence–driven chatbots can provide generally accurate and understandable responses to asthma-related questions. Variability in reliability and accuracy underscores the need for caution in clinical contexts. Artificial intelligence tools may complement but cannot replace professional medical advice in asthma management.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Journal of Allergy and Clinical Immunology-In Practice ALLERGYIMMUNOLOGY-IMMUNOLOGY

CiteScore

11.10

自引率

9.60%

发文量

683

审稿时长

50 days

期刊介绍： JACI: In Practice is an official publication of the American Academy of Allergy, Asthma & Immunology (AAAAI). It is a companion title to The Journal of Allergy and Clinical Immunology, and it aims to provide timely clinical papers, case reports, and management recommendations to clinical allergists and other physicians dealing with allergic and immunologic diseases in their practice. The mission of JACI: In Practice is to offer valid and impactful information that supports evidence-based clinical decisions in the diagnosis and management of asthma, allergies, immunologic conditions, and related diseases. This journal publishes articles on various conditions treated by allergist-immunologists, including food allergy, respiratory disorders (such as asthma, rhinitis, nasal polyps, sinusitis, cough, ABPA, and hypersensitivity pneumonitis), drug allergy, insect sting allergy, anaphylaxis, dermatologic disorders (such as atopic dermatitis, contact dermatitis, urticaria, angioedema, and HAE), immunodeficiency, autoinflammatory syndromes, eosinophilic disorders, and mast cell disorders. The focus of the journal is on providing cutting-edge clinical information that practitioners can use in their everyday practice or to acquire new knowledge and skills for the benefit of their patients. However, mechanistic or translational studies without immediate or near future clinical relevance, as well as animal studies, are not within the scope of the journal.