Development and Validation of a Machine Learning Model for Early Detection of Untreated Infection.

Q4 Medicine
Critical care explorations Pub Date : 2024-10-11 eCollection Date: 2024-10-01 DOI:10.1097/CCE.0000000000001165
Kevin G Buell, Kyle A Carey, Nicole Dussault, William F Parker, Jay Dumanian, Sivasubramanium V Bhavani, Emily R Gilbert, Christopher J Winslow, Nirav S Shah, Majid Afshar, Dana P Edelson, Matthew M Churpek
{"title":"Development and Validation of a Machine Learning Model for Early Detection of Untreated Infection.","authors":"Kevin G Buell, Kyle A Carey, Nicole Dussault, William F Parker, Jay Dumanian, Sivasubramanium V Bhavani, Emily R Gilbert, Christopher J Winslow, Nirav S Shah, Majid Afshar, Dana P Edelson, Matthew M Churpek","doi":"10.1097/CCE.0000000000001165","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Early diagnostic uncertainty for infection causes delays in antibiotic administration in infected patients and unnecessary antibiotic administration in noninfected patients.</p><p><strong>Objective: </strong>To develop a machine learning model for the early detection of untreated infection (eDENTIFI), with the presence of infection determined by clinician chart review.</p><p><strong>Derivation cohort: </strong>Three thousand three hundred fifty-seven adult patients hospitalized between 2006 and 2018 at two health systems in Illinois, United States.</p><p><strong>Validation cohort: </strong>We validated in 1632 patients in a third Illinois health system using area under the receiver operating characteristic curve (AUC).</p><p><strong>Prediction model: </strong>Using a longitudinal discrete-time format, we trained a gradient boosted machine model to predict untreated infection in the next 6 hours using routinely available patient demographics, vital signs, and laboratory results.</p><p><strong>Results: </strong>eDENTIFI had an AUC of 0.80 (95% CI, 0.79-0.81) in the validation cohort and outperformed the systemic inflammatory response syndrome criteria with an AUC of 0.64 (95% CI, 0.64-0.65; p < 0.001). The most important features were body mass index, age, temperature, and heart rate. Using a threshold with a 47.6% sensitivity, eDENTIFI detected infection a median 2.0 hours (interquartile range, 0.9-5.2 hr) before antimicrobial administration, with a negative predictive value of 93.6%. Antibiotic administration guided by eDENTIFI could have decreased unnecessary IV antibiotic administration in noninfected patients by 10.8% absolute or 46.4% relative percentage points compared with clinicians.</p><p><strong>Conclusion: </strong>eDENTIFI could both decrease the time to antimicrobial administration in infected patients and unnecessary antibiotic administration in noninfected patients. Further prospective validation is needed.</p>","PeriodicalId":93957,"journal":{"name":"Critical care explorations","volume":"6 10","pages":"e1165"},"PeriodicalIF":0.0000,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11473064/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Critical care explorations","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1097/CCE.0000000000001165","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/10/1 0:00:00","PubModel":"eCollection","JCR":"Q4","JCRName":"Medicine","Score":null,"Total":0}
引用次数: 0

Abstract

Background: Early diagnostic uncertainty for infection causes delays in antibiotic administration in infected patients and unnecessary antibiotic administration in noninfected patients.

Objective: To develop a machine learning model for the early detection of untreated infection (eDENTIFI), with the presence of infection determined by clinician chart review.

Derivation cohort: Three thousand three hundred fifty-seven adult patients hospitalized between 2006 and 2018 at two health systems in Illinois, United States.

Validation cohort: We validated in 1632 patients in a third Illinois health system using area under the receiver operating characteristic curve (AUC).

Prediction model: Using a longitudinal discrete-time format, we trained a gradient boosted machine model to predict untreated infection in the next 6 hours using routinely available patient demographics, vital signs, and laboratory results.

Results: eDENTIFI had an AUC of 0.80 (95% CI, 0.79-0.81) in the validation cohort and outperformed the systemic inflammatory response syndrome criteria with an AUC of 0.64 (95% CI, 0.64-0.65; p < 0.001). The most important features were body mass index, age, temperature, and heart rate. Using a threshold with a 47.6% sensitivity, eDENTIFI detected infection a median 2.0 hours (interquartile range, 0.9-5.2 hr) before antimicrobial administration, with a negative predictive value of 93.6%. Antibiotic administration guided by eDENTIFI could have decreased unnecessary IV antibiotic administration in noninfected patients by 10.8% absolute or 46.4% relative percentage points compared with clinicians.

Conclusion: eDENTIFI could both decrease the time to antimicrobial administration in infected patients and unnecessary antibiotic administration in noninfected patients. Further prospective validation is needed.

开发并验证用于早期检测未治疗感染的机器学习模型。
背景:感染的早期诊断不确定性会导致感染患者的抗生素用药延迟以及非感染患者不必要的抗生素用药:感染的早期诊断不确定性会导致感染患者延迟使用抗生素,以及非感染患者不必要地使用抗生素:目的:开发一种用于早期检测未经治疗的感染(eDENTIFI)的机器学习模型,通过临床医生的病历审查来确定是否存在感染:结果:在验证队列中,eDENTIFI 的 AUC 为 0.80(95% CI,0.79-0.81),优于全身炎症反应综合征标准,AUC 为 0.64(95% CI,0.64-0.65;P <0.001)。最重要的特征是体重指数、年龄、体温和心率。使用灵敏度为 47.6% 的阈值,eDENTIFI 在使用抗菌药物前中位 2.0 小时(四分位间范围为 0.9-5.2 小时)检测到感染,阴性预测值为 93.6%。与临床医生相比,在 eDENTIFI 的指导下使用抗生素可减少非感染患者不必要的静脉注射抗生素用量,绝对百分比减少 10.8%,相对百分比减少 46.4%。还需要进一步的前瞻性验证。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
5.70
自引率
0.00%
发文量
0
审稿时长
8 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信