Nicholas C Spies, Leah Militello, Christopher W Farnsworth, Joe M El-Khoury, Thomas J S Durant, Mark A Zaydman
{"title":"前瞻性外部验证组合学习方法,灵敏检测基础代谢面板中的静脉注射液污染","authors":"Nicholas C Spies, Leah Militello, Christopher W Farnsworth, Joe M El-Khoury, Thomas J S Durant, Mark A Zaydman","doi":"10.1093/clinchem/hvae168","DOIUrl":null,"url":null,"abstract":"Background Intravenous (IV) fluid contamination within clinical specimens causes an operational burden on the laboratory when detected, and potential patient harm when undetected. Even mild contamination is often sufficient to meaningfully alter results across multiple analytes. A recently reported unsupervised learning approach was more sensitive than routine workflows, but still lacked sensitivity to mild but significant contamination. Here, we leverage ensemble learning to more sensitively detect contaminated results using an approach which is explainable and generalizable across institutions. Methods An ensemble-based machine learning pipeline of general and fluid-specific models was trained on real-world and simulated contamination and internally and externally validated. Benchmarks for performance assessment were derived from in silico simulations, in vitro experiments, and expert review. Fluid-specific regression models estimated contamination severity. SHapley Additive exPlanation (SHAP) values were calculated to explain specimen-level predictions, and algorithmic fairness was evaluated by comparing flag rates across demographic and clinical subgroups. Results The sensitivities, specificities, and Matthews correlation coefficients were 0.858, 0.993, and 0.747 for the internal validation set, and 1.00, 0.980, and 0.387 for the external set. SHAP values provided plausible explanations for dextrose- and ketoacidosis-related hyperglycemia. Flag rates from the pipeline were higher than the current workflow, with improved detection of contamination events expected to exceed allowable limits for measurement error and reference change values. Conclusions An accurate, generalizable, and explainable ensemble-based machine learning pipeline was developed and validated for sensitively detecting IV fluid contamination. Implementing this pipeline would help identify errors that are poorly detected by current clinical workflows and a previously described unsupervised machine learning-based method.","PeriodicalId":10690,"journal":{"name":"Clinical chemistry","volume":"64 1","pages":""},"PeriodicalIF":7.1000,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Prospective and External Validation of an Ensemble Learning Approach to Sensitively Detect Intravenous Fluid Contamination in Basic Metabolic Panels\",\"authors\":\"Nicholas C Spies, Leah Militello, Christopher W Farnsworth, Joe M El-Khoury, Thomas J S Durant, Mark A Zaydman\",\"doi\":\"10.1093/clinchem/hvae168\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Background Intravenous (IV) fluid contamination within clinical specimens causes an operational burden on the laboratory when detected, and potential patient harm when undetected. Even mild contamination is often sufficient to meaningfully alter results across multiple analytes. A recently reported unsupervised learning approach was more sensitive than routine workflows, but still lacked sensitivity to mild but significant contamination. Here, we leverage ensemble learning to more sensitively detect contaminated results using an approach which is explainable and generalizable across institutions. Methods An ensemble-based machine learning pipeline of general and fluid-specific models was trained on real-world and simulated contamination and internally and externally validated. Benchmarks for performance assessment were derived from in silico simulations, in vitro experiments, and expert review. Fluid-specific regression models estimated contamination severity. SHapley Additive exPlanation (SHAP) values were calculated to explain specimen-level predictions, and algorithmic fairness was evaluated by comparing flag rates across demographic and clinical subgroups. Results The sensitivities, specificities, and Matthews correlation coefficients were 0.858, 0.993, and 0.747 for the internal validation set, and 1.00, 0.980, and 0.387 for the external set. SHAP values provided plausible explanations for dextrose- and ketoacidosis-related hyperglycemia. Flag rates from the pipeline were higher than the current workflow, with improved detection of contamination events expected to exceed allowable limits for measurement error and reference change values. Conclusions An accurate, generalizable, and explainable ensemble-based machine learning pipeline was developed and validated for sensitively detecting IV fluid contamination. Implementing this pipeline would help identify errors that are poorly detected by current clinical workflows and a previously described unsupervised machine learning-based method.\",\"PeriodicalId\":10690,\"journal\":{\"name\":\"Clinical chemistry\",\"volume\":\"64 1\",\"pages\":\"\"},\"PeriodicalIF\":7.1000,\"publicationDate\":\"2024-11-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Clinical chemistry\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1093/clinchem/hvae168\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MEDICAL LABORATORY TECHNOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Clinical chemistry","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1093/clinchem/hvae168","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICAL LABORATORY TECHNOLOGY","Score":null,"Total":0}
Prospective and External Validation of an Ensemble Learning Approach to Sensitively Detect Intravenous Fluid Contamination in Basic Metabolic Panels
Background Intravenous (IV) fluid contamination within clinical specimens causes an operational burden on the laboratory when detected, and potential patient harm when undetected. Even mild contamination is often sufficient to meaningfully alter results across multiple analytes. A recently reported unsupervised learning approach was more sensitive than routine workflows, but still lacked sensitivity to mild but significant contamination. Here, we leverage ensemble learning to more sensitively detect contaminated results using an approach which is explainable and generalizable across institutions. Methods An ensemble-based machine learning pipeline of general and fluid-specific models was trained on real-world and simulated contamination and internally and externally validated. Benchmarks for performance assessment were derived from in silico simulations, in vitro experiments, and expert review. Fluid-specific regression models estimated contamination severity. SHapley Additive exPlanation (SHAP) values were calculated to explain specimen-level predictions, and algorithmic fairness was evaluated by comparing flag rates across demographic and clinical subgroups. Results The sensitivities, specificities, and Matthews correlation coefficients were 0.858, 0.993, and 0.747 for the internal validation set, and 1.00, 0.980, and 0.387 for the external set. SHAP values provided plausible explanations for dextrose- and ketoacidosis-related hyperglycemia. Flag rates from the pipeline were higher than the current workflow, with improved detection of contamination events expected to exceed allowable limits for measurement error and reference change values. Conclusions An accurate, generalizable, and explainable ensemble-based machine learning pipeline was developed and validated for sensitively detecting IV fluid contamination. Implementing this pipeline would help identify errors that are poorly detected by current clinical workflows and a previously described unsupervised machine learning-based method.
期刊介绍:
Clinical Chemistry is a peer-reviewed scientific journal that is the premier publication for the science and practice of clinical laboratory medicine. It was established in 1955 and is associated with the Association for Diagnostics & Laboratory Medicine (ADLM).
The journal focuses on laboratory diagnosis and management of patients, and has expanded to include other clinical laboratory disciplines such as genomics, hematology, microbiology, and toxicology. It also publishes articles relevant to clinical specialties including cardiology, endocrinology, gastroenterology, genetics, immunology, infectious diseases, maternal-fetal medicine, neurology, nutrition, oncology, and pediatrics.
In addition to original research, editorials, and reviews, Clinical Chemistry features recurring sections such as clinical case studies, perspectives, podcasts, and Q&A articles. It has the highest impact factor among journals of clinical chemistry, laboratory medicine, pathology, analytical chemistry, transfusion medicine, and clinical microbiology.
The journal is indexed in databases such as MEDLINE and Web of Science.