Using ChatGPT for Kidney Transplantation: Perceived Information Quality by Race and Education Levels

IF 16.4 1区化学 Q1 CHEMISTRY, MULTIDISCIPLINARY

Accounts of Chemical Research Pub Date : 2024-06-27 DOI:10.1111/ctr.15378

Jihye Lee, Jeeyun Park, Hwarang Stephen Han

{"title":"Using ChatGPT for Kidney Transplantation: Perceived Information Quality by Race and Education Levels","authors":"Jihye Lee, Jeeyun Park, Hwarang Stephen Han","doi":"10.1111/ctr.15378","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Kidney transplantation is a complex process requiring extensive preparation and ongoing monitoring. Artificial intelligence (AI)-powered chatbots hold potential for providing accessible health information, but our understanding of their role in offering health advice for kidney transplantation and how individuals assess such advice remains limited. This study investigates how individuals evaluate ChatGPT's responses to kidney transplantation questions in terms of information quality and empathy, focusing on potential differences across race/ethnicity and educational backgrounds.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>We collected Reddit posts (<i>N</i> = 4624) regarding kidney transplantation and selected 86 questions to represent typical clinician inquiries. These questions were used as input prompts for ChatGPT. A total of 565 participants assessed ChatGPT's responses through online surveys, rating information quality and empathy using Likert scales.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Multilevel analyses (<i>N</i> = 2825) show that there is a significant interaction between race/ethnicity and education levels in various measures related to perceived information quality, but not perceived empathy of ChatGPT's responses: accuracy (<i>p</i> < 0.05); authenticity (<i>p</i> < 0.01); believability (<i>p</i> < 0.05); informativeness (<i>p</i> = 0.053); usefulness (<i>p</i> < 0.05); recognizing users’ feelings (<i>p</i> = 0.70) and understanding feelings and situations (<i>p</i> = 0.65). Among non-White individuals, higher education levels predicted higher perceived quality of ChatGPT's responses across all information quality measures. Notably, this trend was reversed for White individuals, where higher education levels led to lower perceived information quality.</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>Our results highlight the importance of developing AI tools sensitive to diverse communication styles and information needs.</p>\n </section>\n </div>","PeriodicalId":1,"journal":{"name":"Accounts of Chemical Research","volume":null,"pages":null},"PeriodicalIF":16.4000,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/ctr.15378","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Accounts of Chemical Research","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/ctr.15378","RegionNum":1,"RegionCategory":"化学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"CHEMISTRY, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Kidney transplantation is a complex process requiring extensive preparation and ongoing monitoring. Artificial intelligence (AI)-powered chatbots hold potential for providing accessible health information, but our understanding of their role in offering health advice for kidney transplantation and how individuals assess such advice remains limited. This study investigates how individuals evaluate ChatGPT's responses to kidney transplantation questions in terms of information quality and empathy, focusing on potential differences across race/ethnicity and educational backgrounds.

Methods

We collected Reddit posts (N = 4624) regarding kidney transplantation and selected 86 questions to represent typical clinician inquiries. These questions were used as input prompts for ChatGPT. A total of 565 participants assessed ChatGPT's responses through online surveys, rating information quality and empathy using Likert scales.

Results

Multilevel analyses (N = 2825) show that there is a significant interaction between race/ethnicity and education levels in various measures related to perceived information quality, but not perceived empathy of ChatGPT's responses: accuracy (p < 0.05); authenticity (p < 0.01); believability (p < 0.05); informativeness (p = 0.053); usefulness (p < 0.05); recognizing users’ feelings (p = 0.70) and understanding feelings and situations (p = 0.65). Among non-White individuals, higher education levels predicted higher perceived quality of ChatGPT's responses across all information quality measures. Notably, this trend was reversed for White individuals, where higher education levels led to lower perceived information quality.

Conclusions

Our results highlight the importance of developing AI tools sensitive to diverse communication styles and information needs.

Abstract Image

查看原文本刊更多论文

使用 ChatGPT 进行肾移植：按种族和教育水平划分的感知信息质量。

背景：肾移植是一个复杂的过程，需要大量准备工作和持续监测。人工智能（AI）驱动的聊天机器人在提供可获取的健康信息方面具有潜力，但我们对其在提供肾移植健康建议方面的作用以及个人如何评估此类建议的了解仍然有限。本研究调查了个人如何从信息质量和移情角度评估 ChatGPT 对肾移植问题的回复，重点关注不同种族/民族和教育背景的潜在差异：我们收集了 Reddit 上有关肾移植的帖子（N = 4624），并选择了 86 个问题来代表典型的临床医生询问。这些问题被用作 ChatGPT 的输入提示。共有 565 名参与者通过在线调查对 ChatGPT 的回复进行了评估，并使用李克特量表对信息质量和同理心进行了评分：多层次分析（N = 2825）显示，种族/民族和教育水平在与 ChatGPT 响应的感知信息质量相关的各种测量中存在显著的交互作用，但在感知共鸣方面却没有显著的交互作用：准确性（p 结论：我们的结果突出了开发人工智能的重要性：我们的研究结果凸显了开发对不同交流方式和信息需求敏感的人工智能工具的重要性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Accounts of Chemical Research 化学-化学综合

CiteScore

31.40

自引率

1.10%

发文量

312

审稿时长

2 months

期刊介绍： Accounts of Chemical Research presents short, concise and critical articles offering easy-to-read overviews of basic research and applications in all areas of chemistry and biochemistry. These short reviews focus on research from the author’s own laboratory and are designed to teach the reader about a research project. In addition, Accounts of Chemical Research publishes commentaries that give an informed opinion on a current research problem. Special Issues online are devoted to a single topic of unusual activity and significance. Accounts of Chemical Research replaces the traditional article abstract with an article "Conspectus." These entries synopsize the research affording the reader a closer look at the content and significance of an article. Through this provision of a more detailed description of the article contents, the Conspectus enhances the article's discoverability by search engines and the exposure for the research.