Generative artificial intelligence responses to patient messages in the electronic health record: early lessons learned

IF 2.5 Q2 HEALTH CARE SCIENCES & SERVICES
Sally L Baxter, Christopher Longhurst, Marlene Millen, Amy M Sitapati, Ming Tai-Seale
{"title":"Generative artificial intelligence responses to patient messages in the electronic health record: early lessons learned","authors":"Sally L Baxter, Christopher Longhurst, Marlene Millen, Amy M Sitapati, Ming Tai-Seale","doi":"10.1093/jamiaopen/ooae028","DOIUrl":null,"url":null,"abstract":"Abstract Background Electronic health record (EHR)-based patient messages can contribute to burnout. Messages with a negative tone are particularly challenging to address. In this perspective, we describe our initial evaluation of large language model (LLM)-generated responses to negative EHR patient messages and contend that using LLMs to generate initial drafts may be feasible, although refinement will be needed. Methods A retrospective sample (n = 50) of negative patient messages was extracted from a health system EHR, de-identified, and inputted into an LLM (ChatGPT). Qualitative analyses were conducted to compare LLM responses to actual care team responses. Results Some LLM-generated draft responses varied from human responses in relational connection, informational content, and recommendations for next steps. Occasionally, the LLM draft responses could have potentially escalated emotionally charged conversations. Conclusion Further work is needed to optimize the use of LLMs for responding to negative patient messages in the EHR.","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":null,"pages":null},"PeriodicalIF":2.5000,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"JAMIA Open","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1093/jamiaopen/ooae028","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

Abstract

Abstract Background Electronic health record (EHR)-based patient messages can contribute to burnout. Messages with a negative tone are particularly challenging to address. In this perspective, we describe our initial evaluation of large language model (LLM)-generated responses to negative EHR patient messages and contend that using LLMs to generate initial drafts may be feasible, although refinement will be needed. Methods A retrospective sample (n = 50) of negative patient messages was extracted from a health system EHR, de-identified, and inputted into an LLM (ChatGPT). Qualitative analyses were conducted to compare LLM responses to actual care team responses. Results Some LLM-generated draft responses varied from human responses in relational connection, informational content, and recommendations for next steps. Occasionally, the LLM draft responses could have potentially escalated emotionally charged conversations. Conclusion Further work is needed to optimize the use of LLMs for responding to negative patient messages in the EHR.
对电子健康记录中患者信息的人工智能生成反应:早期经验教训
摘要 背景 基于电子健康记录(EHR)的患者信息可能会导致职业倦怠。带有负面语气的信息尤其具有挑战性。在这一视角中,我们描述了我们对由大型语言模型(LLM)生成的电子病历患者负面信息回复的初步评估,并认为使用 LLM 生成初稿可能是可行的,尽管还需要改进。方法 从医疗系统的电子病历中提取负面患者信息的回顾性样本(n = 50),去除身份标识并输入 LLM (ChatGPT)。对 LLM 的回复与实际护理团队的回复进行了定性分析比较。结果 LLM 生成的一些回复草稿在关系连接、信息内容和下一步建议方面与人工回复存在差异。有时,LLM 的回复草稿可能会使情绪化的对话升级。结论 要优化使用 LLM 来回应电子病历中患者的负面信息,还需要进一步的工作。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
JAMIA Open
JAMIA Open Medicine-Health Informatics
CiteScore
4.10
自引率
4.80%
发文量
102
审稿时长
16 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信