前瞻性外部验证组合学习方法,灵敏检测基础代谢面板中的静脉注射液污染

IF 7.1 2区 医学 Q1 MEDICAL LABORATORY TECHNOLOGY
Nicholas C Spies, Leah Militello, Christopher W Farnsworth, Joe M El-Khoury, Thomas J S Durant, Mark A Zaydman
{"title":"前瞻性外部验证组合学习方法,灵敏检测基础代谢面板中的静脉注射液污染","authors":"Nicholas C Spies, Leah Militello, Christopher W Farnsworth, Joe M El-Khoury, Thomas J S Durant, Mark A Zaydman","doi":"10.1093/clinchem/hvae168","DOIUrl":null,"url":null,"abstract":"Background Intravenous (IV) fluid contamination within clinical specimens causes an operational burden on the laboratory when detected, and potential patient harm when undetected. Even mild contamination is often sufficient to meaningfully alter results across multiple analytes. A recently reported unsupervised learning approach was more sensitive than routine workflows, but still lacked sensitivity to mild but significant contamination. Here, we leverage ensemble learning to more sensitively detect contaminated results using an approach which is explainable and generalizable across institutions. Methods An ensemble-based machine learning pipeline of general and fluid-specific models was trained on real-world and simulated contamination and internally and externally validated. Benchmarks for performance assessment were derived from in silico simulations, in vitro experiments, and expert review. Fluid-specific regression models estimated contamination severity. SHapley Additive exPlanation (SHAP) values were calculated to explain specimen-level predictions, and algorithmic fairness was evaluated by comparing flag rates across demographic and clinical subgroups. Results The sensitivities, specificities, and Matthews correlation coefficients were 0.858, 0.993, and 0.747 for the internal validation set, and 1.00, 0.980, and 0.387 for the external set. SHAP values provided plausible explanations for dextrose- and ketoacidosis-related hyperglycemia. Flag rates from the pipeline were higher than the current workflow, with improved detection of contamination events expected to exceed allowable limits for measurement error and reference change values. Conclusions An accurate, generalizable, and explainable ensemble-based machine learning pipeline was developed and validated for sensitively detecting IV fluid contamination. Implementing this pipeline would help identify errors that are poorly detected by current clinical workflows and a previously described unsupervised machine learning-based method.","PeriodicalId":10690,"journal":{"name":"Clinical chemistry","volume":"64 1","pages":""},"PeriodicalIF":7.1000,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Prospective and External Validation of an Ensemble Learning Approach to Sensitively Detect Intravenous Fluid Contamination in Basic Metabolic Panels\",\"authors\":\"Nicholas C Spies, Leah Militello, Christopher W Farnsworth, Joe M El-Khoury, Thomas J S Durant, Mark A Zaydman\",\"doi\":\"10.1093/clinchem/hvae168\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Background Intravenous (IV) fluid contamination within clinical specimens causes an operational burden on the laboratory when detected, and potential patient harm when undetected. Even mild contamination is often sufficient to meaningfully alter results across multiple analytes. A recently reported unsupervised learning approach was more sensitive than routine workflows, but still lacked sensitivity to mild but significant contamination. Here, we leverage ensemble learning to more sensitively detect contaminated results using an approach which is explainable and generalizable across institutions. Methods An ensemble-based machine learning pipeline of general and fluid-specific models was trained on real-world and simulated contamination and internally and externally validated. Benchmarks for performance assessment were derived from in silico simulations, in vitro experiments, and expert review. Fluid-specific regression models estimated contamination severity. SHapley Additive exPlanation (SHAP) values were calculated to explain specimen-level predictions, and algorithmic fairness was evaluated by comparing flag rates across demographic and clinical subgroups. Results The sensitivities, specificities, and Matthews correlation coefficients were 0.858, 0.993, and 0.747 for the internal validation set, and 1.00, 0.980, and 0.387 for the external set. SHAP values provided plausible explanations for dextrose- and ketoacidosis-related hyperglycemia. Flag rates from the pipeline were higher than the current workflow, with improved detection of contamination events expected to exceed allowable limits for measurement error and reference change values. Conclusions An accurate, generalizable, and explainable ensemble-based machine learning pipeline was developed and validated for sensitively detecting IV fluid contamination. Implementing this pipeline would help identify errors that are poorly detected by current clinical workflows and a previously described unsupervised machine learning-based method.\",\"PeriodicalId\":10690,\"journal\":{\"name\":\"Clinical chemistry\",\"volume\":\"64 1\",\"pages\":\"\"},\"PeriodicalIF\":7.1000,\"publicationDate\":\"2024-11-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Clinical chemistry\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1093/clinchem/hvae168\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MEDICAL LABORATORY TECHNOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Clinical chemistry","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1093/clinchem/hvae168","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICAL LABORATORY TECHNOLOGY","Score":null,"Total":0}
引用次数: 0

摘要

背景 临床标本中的静脉注射液(IV)污染一旦被检测到,就会给实验室带来操作负担,而如果未被检测到,则可能对患者造成伤害。即使是轻度污染,也往往足以对多种分析物的检测结果造成有意义的改变。最近报道的一种无监督学习方法比常规工作流程更灵敏,但对轻微但严重的污染仍然缺乏敏感性。在这里,我们利用集合学习来更灵敏地检测污染结果,这种方法可以在不同机构之间进行解释和推广。方法 在真实世界和模拟污染的基础上,对通用模型和特定液体模型进行了基于集合的机器学习管道训练,并进行了内部和外部验证。性能评估基准来自硅学模拟、体外实验和专家评审。特定流体回归模型估计了污染严重程度。通过计算 SHapley Additive exPlanation(SHAP)值来解释标本级预测,并通过比较不同人群和临床亚群的标记率来评估算法的公平性。结果 内部验证集的灵敏度、特异性和马修斯相关系数分别为 0.858、0.993 和 0.747,外部验证集的灵敏度、特异性和马修斯相关系数分别为 1.00、0.980 和 0.387。SHAP 值为葡萄糖和酮症酸中毒相关的高血糖提供了合理的解释。流水线的标记率高于当前工作流程,对超出测量误差和参考变化值允许范围的污染事件的检测能力也有所提高。结论 为灵敏检测静脉输液污染,开发并验证了一种准确、可推广且可解释的基于集合的机器学习管道。实施该管道将有助于识别当前临床工作流程和之前描述的基于无监督机器学习方法难以检测到的错误。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Prospective and External Validation of an Ensemble Learning Approach to Sensitively Detect Intravenous Fluid Contamination in Basic Metabolic Panels
Background Intravenous (IV) fluid contamination within clinical specimens causes an operational burden on the laboratory when detected, and potential patient harm when undetected. Even mild contamination is often sufficient to meaningfully alter results across multiple analytes. A recently reported unsupervised learning approach was more sensitive than routine workflows, but still lacked sensitivity to mild but significant contamination. Here, we leverage ensemble learning to more sensitively detect contaminated results using an approach which is explainable and generalizable across institutions. Methods An ensemble-based machine learning pipeline of general and fluid-specific models was trained on real-world and simulated contamination and internally and externally validated. Benchmarks for performance assessment were derived from in silico simulations, in vitro experiments, and expert review. Fluid-specific regression models estimated contamination severity. SHapley Additive exPlanation (SHAP) values were calculated to explain specimen-level predictions, and algorithmic fairness was evaluated by comparing flag rates across demographic and clinical subgroups. Results The sensitivities, specificities, and Matthews correlation coefficients were 0.858, 0.993, and 0.747 for the internal validation set, and 1.00, 0.980, and 0.387 for the external set. SHAP values provided plausible explanations for dextrose- and ketoacidosis-related hyperglycemia. Flag rates from the pipeline were higher than the current workflow, with improved detection of contamination events expected to exceed allowable limits for measurement error and reference change values. Conclusions An accurate, generalizable, and explainable ensemble-based machine learning pipeline was developed and validated for sensitively detecting IV fluid contamination. Implementing this pipeline would help identify errors that are poorly detected by current clinical workflows and a previously described unsupervised machine learning-based method.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Clinical chemistry
Clinical chemistry 医学-医学实验技术
CiteScore
11.30
自引率
4.30%
发文量
212
审稿时长
1.7 months
期刊介绍: Clinical Chemistry is a peer-reviewed scientific journal that is the premier publication for the science and practice of clinical laboratory medicine. It was established in 1955 and is associated with the Association for Diagnostics & Laboratory Medicine (ADLM). The journal focuses on laboratory diagnosis and management of patients, and has expanded to include other clinical laboratory disciplines such as genomics, hematology, microbiology, and toxicology. It also publishes articles relevant to clinical specialties including cardiology, endocrinology, gastroenterology, genetics, immunology, infectious diseases, maternal-fetal medicine, neurology, nutrition, oncology, and pediatrics. In addition to original research, editorials, and reviews, Clinical Chemistry features recurring sections such as clinical case studies, perspectives, podcasts, and Q&A articles. It has the highest impact factor among journals of clinical chemistry, laboratory medicine, pathology, analytical chemistry, transfusion medicine, and clinical microbiology. The journal is indexed in databases such as MEDLINE and Web of Science.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信