ECG-LM:用大语言模型理解心电图。

Health data science Pub Date : 2025-02-04 eCollection Date: 2025-01-01 DOI:10.34133/hds.0221
Kai Yang, Massimo Hong, Jiahuan Zhang, Yizhen Luo, Suyuan Zhao, Ou Zhang, Xiaomao Yu, Jiawen Zhou, Liuqing Yang, Ping Zhang, Mu Qiao, Zaiqing Nie
{"title":"ECG-LM:用大语言模型理解心电图。","authors":"Kai Yang, Massimo Hong, Jiahuan Zhang, Yizhen Luo, Suyuan Zhao, Ou Zhang, Xiaomao Yu, Jiawen Zhou, Liuqing Yang, Ping Zhang, Mu Qiao, Zaiqing Nie","doi":"10.34133/hds.0221","DOIUrl":null,"url":null,"abstract":"<p><p><b>Background:</b> The electrocardiogram (ECG) is a valuable, noninvasive tool for monitoring heart-related conditions, providing critical insights. However, the interpretation of ECG data alongside patient information demands substantial medical expertise and resources. While deep learning methods help streamline this process, they often fall short in integrating patient data with ECG readings and do not provide the nuanced clinical suggestions and insights necessary for accurate diagnosis. <b>Methods:</b> Although recent advancements in multi-modal large language modeling have propelled their application scope beyond the natural language processing domain, their applicability to ECG processing remains largely unexplored, partly due to the lack of text-ECG data. To this end, we develop ECG-Language Model (ECG-LM), the first multi-modal large language model able to process natural language and understand ECG signals. The model employs a specialized ECG encoder that transforms raw ECG signals into a high-dimensional feature space, which is then aligned with the textual feature space derived from the large language model. To address the scarcity of text-ECG data, we generated text-ECG pairs by leveraging detailed ECG pattern descriptions from medical guidelines, creating a robust dataset for pre-training ECG-LM. Additionally, we fine-tune ECG-LM with public clinical conversation datasets and build an additional supervised fine-tuning dataset based on real clinical data from the hospital, aiming to provide a more comprehensive and customized user experience. <b>Results:</b> ECG-LM outperforms existing few-shot and zero-shot solutions in cardiovascular disease detection across all 3 tasks (diagnostic, rhythm, and form) while also demonstrating strong potential in ECG-related question answering. <b>Conclusions:</b> The results across various tasks demonstrate that ECG-LM effectively captures the intricate features of ECGs, showcasing its versatility in applications such as disease prediction and advanced question answering.</p>","PeriodicalId":73207,"journal":{"name":"Health data science","volume":"5 ","pages":"0221"},"PeriodicalIF":0.0000,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11791404/pdf/","citationCount":"0","resultStr":"{\"title\":\"ECG-LM: Understanding Electrocardiogram with a Large Language Model.\",\"authors\":\"Kai Yang, Massimo Hong, Jiahuan Zhang, Yizhen Luo, Suyuan Zhao, Ou Zhang, Xiaomao Yu, Jiawen Zhou, Liuqing Yang, Ping Zhang, Mu Qiao, Zaiqing Nie\",\"doi\":\"10.34133/hds.0221\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p><b>Background:</b> The electrocardiogram (ECG) is a valuable, noninvasive tool for monitoring heart-related conditions, providing critical insights. However, the interpretation of ECG data alongside patient information demands substantial medical expertise and resources. While deep learning methods help streamline this process, they often fall short in integrating patient data with ECG readings and do not provide the nuanced clinical suggestions and insights necessary for accurate diagnosis. <b>Methods:</b> Although recent advancements in multi-modal large language modeling have propelled their application scope beyond the natural language processing domain, their applicability to ECG processing remains largely unexplored, partly due to the lack of text-ECG data. To this end, we develop ECG-Language Model (ECG-LM), the first multi-modal large language model able to process natural language and understand ECG signals. The model employs a specialized ECG encoder that transforms raw ECG signals into a high-dimensional feature space, which is then aligned with the textual feature space derived from the large language model. To address the scarcity of text-ECG data, we generated text-ECG pairs by leveraging detailed ECG pattern descriptions from medical guidelines, creating a robust dataset for pre-training ECG-LM. Additionally, we fine-tune ECG-LM with public clinical conversation datasets and build an additional supervised fine-tuning dataset based on real clinical data from the hospital, aiming to provide a more comprehensive and customized user experience. <b>Results:</b> ECG-LM outperforms existing few-shot and zero-shot solutions in cardiovascular disease detection across all 3 tasks (diagnostic, rhythm, and form) while also demonstrating strong potential in ECG-related question answering. <b>Conclusions:</b> The results across various tasks demonstrate that ECG-LM effectively captures the intricate features of ECGs, showcasing its versatility in applications such as disease prediction and advanced question answering.</p>\",\"PeriodicalId\":73207,\"journal\":{\"name\":\"Health data science\",\"volume\":\"5 \",\"pages\":\"0221\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-02-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11791404/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Health data science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.34133/hds.0221\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2025/1/1 0:00:00\",\"PubModel\":\"eCollection\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Health data science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.34133/hds.0221","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/1/1 0:00:00","PubModel":"eCollection","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

背景:心电图(ECG)是一种有价值的、无创的监测心脏相关疾病的工具,提供了重要的见解。然而,心电图数据和患者信息的解释需要大量的医学专业知识和资源。虽然深度学习方法有助于简化这一过程,但它们在将患者数据与ECG读数整合方面往往存在不足,并且不能提供准确诊断所需的细致入微的临床建议和见解。方法:尽管近年来多模态大语言建模的进展使其应用范围超出了自然语言处理领域,但由于缺乏文本-心电数据,其在心电处理中的适用性在很大程度上仍未得到探索。为此,我们开发了ECG-语言模型(ECG- lm),这是第一个能够处理自然语言并理解心电信号的多模态大型语言模型。该模型采用专门的心电编码器,将原始心电信号转换为高维特征空间,然后与大语言模型导出的文本特征空间对齐。为了解决文本-ECG数据的稀缺性,我们利用医疗指南中详细的ECG模式描述生成了文本-ECG对,创建了一个用于预训练ECG- lm的鲁棒数据集。此外,我们使用公开的临床会话数据集对ECG-LM进行微调,并基于医院的真实临床数据构建额外的监督微调数据集,旨在提供更全面和定制的用户体验。结果:ECG-LM在心血管疾病检测的所有3个任务(诊断、节律和形式)中都优于现有的少射和零射解决方案,同时在ecg相关的问题回答中也显示出强大的潜力。结论:各种任务的结果表明,ECG-LM有效地捕获了心电图的复杂特征,展示了其在疾病预测和高级问题回答等应用中的多功能性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
ECG-LM: Understanding Electrocardiogram with a Large Language Model.

Background: The electrocardiogram (ECG) is a valuable, noninvasive tool for monitoring heart-related conditions, providing critical insights. However, the interpretation of ECG data alongside patient information demands substantial medical expertise and resources. While deep learning methods help streamline this process, they often fall short in integrating patient data with ECG readings and do not provide the nuanced clinical suggestions and insights necessary for accurate diagnosis. Methods: Although recent advancements in multi-modal large language modeling have propelled their application scope beyond the natural language processing domain, their applicability to ECG processing remains largely unexplored, partly due to the lack of text-ECG data. To this end, we develop ECG-Language Model (ECG-LM), the first multi-modal large language model able to process natural language and understand ECG signals. The model employs a specialized ECG encoder that transforms raw ECG signals into a high-dimensional feature space, which is then aligned with the textual feature space derived from the large language model. To address the scarcity of text-ECG data, we generated text-ECG pairs by leveraging detailed ECG pattern descriptions from medical guidelines, creating a robust dataset for pre-training ECG-LM. Additionally, we fine-tune ECG-LM with public clinical conversation datasets and build an additional supervised fine-tuning dataset based on real clinical data from the hospital, aiming to provide a more comprehensive and customized user experience. Results: ECG-LM outperforms existing few-shot and zero-shot solutions in cardiovascular disease detection across all 3 tasks (diagnostic, rhythm, and form) while also demonstrating strong potential in ECG-related question answering. Conclusions: The results across various tasks demonstrate that ECG-LM effectively captures the intricate features of ECGs, showcasing its versatility in applications such as disease prediction and advanced question answering.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
3.70
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信