Development and Validation of a Machine Learning COVID-19 Veteran (COVet) Deterioration Risk Score.

Q4 Medicine
Critical care explorations Pub Date : 2024-07-19 eCollection Date: 2024-07-01 DOI:10.1097/CCE.0000000000001116
Sushant Govindan, Alexandra Spicer, Matthew Bearce, Richard S Schaefer, Andrea Uhl, Gil Alterovitz, Michael J Kim, Kyle A Carey, Nirav S Shah, Christopher Winslow, Emily Gilbert, Anne Stey, Alan M Weiss, Devendra Amin, George Karway, Jennie Martin, Dana P Edelson, Matthew M Churpek
{"title":"Development and Validation of a Machine Learning COVID-19 Veteran (COVet) Deterioration Risk Score.","authors":"Sushant Govindan, Alexandra Spicer, Matthew Bearce, Richard S Schaefer, Andrea Uhl, Gil Alterovitz, Michael J Kim, Kyle A Carey, Nirav S Shah, Christopher Winslow, Emily Gilbert, Anne Stey, Alan M Weiss, Devendra Amin, George Karway, Jennie Martin, Dana P Edelson, Matthew M Churpek","doi":"10.1097/CCE.0000000000001116","DOIUrl":null,"url":null,"abstract":"<p><strong>Background and objective: </strong>To develop the COVid Veteran (COVet) score for clinical deterioration in Veterans hospitalized with COVID-19 and further validate this model in both Veteran and non-Veteran samples. No such score has been derived and validated while incorporating a Veteran sample.</p><p><strong>Derivation cohort: </strong>Adults (age ≥ 18 yr) hospitalized outside the ICU with a diagnosis of COVID-19 for model development to the Veterans Health Administration (VHA) (n = 80 hospitals).</p><p><strong>Validation cohort: </strong>External validation occurred in a VHA cohort of 34 hospitals, as well as six non-Veteran health systems for further external validation (n = 21 hospitals) between 2020 and 2023.</p><p><strong>Prediction model: </strong>eXtreme Gradient Boosting machine learning methods were used, and performance was assessed using the area under the receiver operating characteristic curve and compared with the National Early Warning Score (NEWS). The primary outcome was transfer to the ICU or death within 24 hours of each new variable observation. Model predictor variables included demographics, vital signs, structured flowsheet data, and laboratory values.</p><p><strong>Results: </strong>A total of 96,908 admissions occurred during the study period, of which 59,897 were in the Veteran sample and 37,011 were in the non-Veteran sample. During external validation in the Veteran sample, the model demonstrated excellent discrimination, with an area under the receiver operating characteristic curve of 0.88. This was significantly higher than NEWS (0.79; p < 0.01). In the non-Veteran sample, the model also demonstrated excellent discrimination (0.86 vs. 0.79 for NEWS; p < 0.01). The top three variables of importance were eosinophil percentage, mean oxygen saturation in the prior 24-hour period, and worst mental status in the prior 24-hour period.</p><p><strong>Conclusions: </strong>We used machine learning methods to develop and validate a highly accurate early warning score in both Veterans and non-Veterans hospitalized with COVID-19. The model could lead to earlier identification and therapy, which may improve outcomes.</p>","PeriodicalId":93957,"journal":{"name":"Critical care explorations","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11262818/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Critical care explorations","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1097/CCE.0000000000001116","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/7/1 0:00:00","PubModel":"eCollection","JCR":"Q4","JCRName":"Medicine","Score":null,"Total":0}
引用次数: 0

Abstract

Background and objective: To develop the COVid Veteran (COVet) score for clinical deterioration in Veterans hospitalized with COVID-19 and further validate this model in both Veteran and non-Veteran samples. No such score has been derived and validated while incorporating a Veteran sample.

Derivation cohort: Adults (age ≥ 18 yr) hospitalized outside the ICU with a diagnosis of COVID-19 for model development to the Veterans Health Administration (VHA) (n = 80 hospitals).

Validation cohort: External validation occurred in a VHA cohort of 34 hospitals, as well as six non-Veteran health systems for further external validation (n = 21 hospitals) between 2020 and 2023.

Prediction model: eXtreme Gradient Boosting machine learning methods were used, and performance was assessed using the area under the receiver operating characteristic curve and compared with the National Early Warning Score (NEWS). The primary outcome was transfer to the ICU or death within 24 hours of each new variable observation. Model predictor variables included demographics, vital signs, structured flowsheet data, and laboratory values.

Results: A total of 96,908 admissions occurred during the study period, of which 59,897 were in the Veteran sample and 37,011 were in the non-Veteran sample. During external validation in the Veteran sample, the model demonstrated excellent discrimination, with an area under the receiver operating characteristic curve of 0.88. This was significantly higher than NEWS (0.79; p < 0.01). In the non-Veteran sample, the model also demonstrated excellent discrimination (0.86 vs. 0.79 for NEWS; p < 0.01). The top three variables of importance were eosinophil percentage, mean oxygen saturation in the prior 24-hour period, and worst mental status in the prior 24-hour period.

Conclusions: We used machine learning methods to develop and validate a highly accurate early warning score in both Veterans and non-Veterans hospitalized with COVID-19. The model could lead to earlier identification and therapy, which may improve outcomes.

机器学习 COVID-19 退伍军人 (COVet) 病情恶化风险评分的开发与验证。
背景和目的:针对因 COVID-19 而住院的退伍军人的临床恶化情况制定 COVid 退伍军人(COVet)评分,并在退伍军人和非退伍军人样本中进一步验证该模型。衍生队列:在重症监护室外住院并诊断为 COVID-19 的成人(年龄≥ 18 岁),用于退伍军人健康管理局 (VHA) 的模型开发(n = 80 家医院)。验证队列:外部验证在退伍军人健康管理局的 34 家医院队列中进行,并在 2020 年至 2023 年期间在 6 家非退伍军人健康系统中进行进一步外部验证(n = 21 家医院)。预测模型:使用梯度提升机器学习方法,使用接收器操作特征曲线下面积评估性能,并与国家预警评分(NEWS)进行比较。主要结果是在观察到每个新变量后的 24 小时内转入重症监护室或死亡。模型预测变量包括人口统计学、生命体征、结构化流程表数据和实验室值:研究期间共有96908人入院,其中退伍军人样本为59897人,非退伍军人样本为37011人。在退伍军人样本的外部验证中,该模型表现出了极佳的辨别能力,接收者操作特征曲线下面积为 0.88。这明显高于 "新闻"(0.79;P < 0.01)。在非退伍军人样本中,该模型也表现出了极佳的辨别能力(0.86,而 NEWS 为 0.79;p < 0.01)。最重要的三个变量是嗜酸性粒细胞百分比、前24小时的平均血氧饱和度和前24小时的最差精神状态:我们使用机器学习方法开发并验证了一种高度准确的预警评分,适用于因 COVID-19 而住院的退伍军人和非退伍军人。该模型可以提前识别和治疗,从而改善预后。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
5.70
自引率
0.00%
发文量
0
审稿时长
8 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信