Prediction of Moderate-to-Severe Sepsis-Associated Acute Kidney Injury Using a Dual-Timepoint Machine Learning Model: Development, Multiregional Validation, and Clinical Deployment Study.
Xinbo Ge, Weiwei Chen, Jianshan Shi, Jiaqiang Zhang, Hao Tai, Ying Zhang, Biao Wang, Wei Liu, Song Chen, Huirui Han
{"title":"Prediction of Moderate-to-Severe Sepsis-Associated Acute Kidney Injury Using a Dual-Timepoint Machine Learning Model: Development, Multiregional Validation, and Clinical Deployment Study.","authors":"Xinbo Ge, Weiwei Chen, Jianshan Shi, Jiaqiang Zhang, Hao Tai, Ying Zhang, Biao Wang, Wei Liu, Song Chen, Huirui Han","doi":"10.2196/73840","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Sepsis-associated acute kidney injury (SA-AKI) is a frequent and life-threatening complication in patients in the intensive care unit (ICU), significantly increasing both mortality rates and the risk of chronic kidney dysfunction. However, existing prediction models have often focused on overall risk and lack severity-based stratification, which limits their clinical applicability.</p><p><strong>Objective: </strong>This study aimed to identify critical time points in SA-AKI progression development and validate dynamic, stratified machine learning prediction models for moderate-to-severe (Kidney Disease: Improving Global Outcomes guideline stages 2-3) SA-AKI through multicenter, multiregional external validation, ultimately deploying them as publicly accessible, interpretable clinical decision support tools.</p><p><strong>Methods: </strong>This study used three independent ICU databases: Medical Information Mart for Intensive Care-IV v3.0 (n=12,842; model development and internal validation), electronic ICU collaborative research database v2.0 (n=15,767; North American multicenter external validation), and the First Affiliated Hospital of Hainan Medical University ICU (n=210; Chinese single-center external validation). We identified 48 hours (acute phase) and 7 days (subacute phase) as critical time points. Based on clinical data from the first 24 hours of ICU admission, we used a two-stage feature selection process combining light gradient boosting machine (LightGBM) and Shapley additive explanation (SHAP) cross-validation analysis with clinical expert review, followed by modeling using 8 machine learning algorithms. The optimal model was selected based on the area under the receiver operating characteristic curve (AUC), calibration curves, and decision curve analysis. Internal validation used 5-fold cross-validation, while external validation and subgroup analyses assessed generalizability across different regions and populations. SHAP values and partial dependence plots were used to interpret the influence of key features on predictions.</p><p><strong>Results: </strong>Our dual-timepoint LightGBM model demonstrated robust predictive performance. For the 48-hour prediction task, the model achieved an AUC of 0.839 (95% CI 0.824-0.854) in the internal test set, with AUCs of 0.770 (95% CI 0.762-0.779) and 0.793 (95% CI 0.726-0.856) in the external validation cohorts, respectively. For the 7-day prediction task, the corresponding AUCs across the three cohorts were 0.834 (95% CI 0.818-0.850), 0.720 (95% CI 0.711-0.729), and 0.773 (95% CI 0.687-0.851), respectively. Subgroup analyses confirmed robust model performance across different age, gender, and comorbidity subgroups. SHAP analysis identified urine output, mechanical ventilation, Sequential Organ Failure Assessment score, creatinine, Glasgow Coma Scale score, and nephrotoxic drug use as core predictive features. Decision curve analysis confirmed that LightGBM provided consistent clinical benefit across different threshold ranges. The optimal LightGBM model was deployed as a publicly accessible web-based prediction app with integrated SHAP interpretability.</p><p><strong>Conclusions: </strong>This study developed and validated a dynamic, stratified prediction system that provides stage-specific risk assessment for moderate-to-severe SA-AKI. The system underwent rigorous multiregional, multicenter validation and was translated into an interpretable clinical decision support tool, providing a scientific foundation for precision management.</p>","PeriodicalId":16337,"journal":{"name":"Journal of Medical Internet Research","volume":"27 ","pages":"e73840"},"PeriodicalIF":6.0000,"publicationDate":"2025-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Medical Internet Research","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.2196/73840","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Sepsis-associated acute kidney injury (SA-AKI) is a frequent and life-threatening complication in patients in the intensive care unit (ICU), significantly increasing both mortality rates and the risk of chronic kidney dysfunction. However, existing prediction models have often focused on overall risk and lack severity-based stratification, which limits their clinical applicability.
Objective: This study aimed to identify critical time points in SA-AKI progression development and validate dynamic, stratified machine learning prediction models for moderate-to-severe (Kidney Disease: Improving Global Outcomes guideline stages 2-3) SA-AKI through multicenter, multiregional external validation, ultimately deploying them as publicly accessible, interpretable clinical decision support tools.
Methods: This study used three independent ICU databases: Medical Information Mart for Intensive Care-IV v3.0 (n=12,842; model development and internal validation), electronic ICU collaborative research database v2.0 (n=15,767; North American multicenter external validation), and the First Affiliated Hospital of Hainan Medical University ICU (n=210; Chinese single-center external validation). We identified 48 hours (acute phase) and 7 days (subacute phase) as critical time points. Based on clinical data from the first 24 hours of ICU admission, we used a two-stage feature selection process combining light gradient boosting machine (LightGBM) and Shapley additive explanation (SHAP) cross-validation analysis with clinical expert review, followed by modeling using 8 machine learning algorithms. The optimal model was selected based on the area under the receiver operating characteristic curve (AUC), calibration curves, and decision curve analysis. Internal validation used 5-fold cross-validation, while external validation and subgroup analyses assessed generalizability across different regions and populations. SHAP values and partial dependence plots were used to interpret the influence of key features on predictions.
Results: Our dual-timepoint LightGBM model demonstrated robust predictive performance. For the 48-hour prediction task, the model achieved an AUC of 0.839 (95% CI 0.824-0.854) in the internal test set, with AUCs of 0.770 (95% CI 0.762-0.779) and 0.793 (95% CI 0.726-0.856) in the external validation cohorts, respectively. For the 7-day prediction task, the corresponding AUCs across the three cohorts were 0.834 (95% CI 0.818-0.850), 0.720 (95% CI 0.711-0.729), and 0.773 (95% CI 0.687-0.851), respectively. Subgroup analyses confirmed robust model performance across different age, gender, and comorbidity subgroups. SHAP analysis identified urine output, mechanical ventilation, Sequential Organ Failure Assessment score, creatinine, Glasgow Coma Scale score, and nephrotoxic drug use as core predictive features. Decision curve analysis confirmed that LightGBM provided consistent clinical benefit across different threshold ranges. The optimal LightGBM model was deployed as a publicly accessible web-based prediction app with integrated SHAP interpretability.
Conclusions: This study developed and validated a dynamic, stratified prediction system that provides stage-specific risk assessment for moderate-to-severe SA-AKI. The system underwent rigorous multiregional, multicenter validation and was translated into an interpretable clinical decision support tool, providing a scientific foundation for precision management.
背景:脓毒症相关急性肾损伤(SA-AKI)是重症监护病房(ICU)患者中一种常见且危及生命的并发症,显著增加死亡率和慢性肾功能障碍的风险。然而,现有的预测模型往往侧重于整体风险,缺乏基于严重程度的分层,这限制了其临床适用性。目的:本研究旨在通过多中心、多地区的外部验证,确定SA-AKI进展发展的关键时间点,并验证中重度(肾脏疾病:改善全球结局指南阶段2-3)SA-AKI的动态分层机器学习预测模型,最终将其部署为可公开访问的、可解释的临床决策支持工具。方法:本研究使用3个独立的ICU数据库:重症监护医学信息市场- iv v3.0 (n=12,842,模型开发和内部验证)、电子ICU协同研究数据库v2.0 (n=15,767,北美多中心外部验证)和海南医科大学第一附属医院ICU (n=210,中国单中心外部验证)。我们确定48小时(急性期)和7天(亚急性期)为关键时间点。基于ICU入院前24小时的临床数据,我们采用两阶段特征选择过程,结合光梯度增强机(LightGBM)和Shapley加性解释(SHAP)交叉验证分析和临床专家评审,然后使用8种机器学习算法建模。根据受试者工作特性曲线(AUC)下面积、标定曲线和决策曲线分析选择最优模型。内部验证使用5倍交叉验证,而外部验证和亚组分析评估了不同地区和人群的普遍性。使用SHAP值和部分依赖图来解释关键特征对预测的影响。结果:我们的双时间点LightGBM模型显示出稳健的预测性能。对于48小时的预测任务,该模型在内部测试集的AUC为0.839 (95% CI 0.824-0.854),在外部验证队列的AUC分别为0.770 (95% CI 0.762-0.779)和0.793 (95% CI 0.726-0.856)。对于7天预测任务,三个队列对应的auc分别为0.834 (95% CI 0.818-0.850)、0.720 (95% CI 0.711-0.729)和0.773 (95% CI 0.687-0.851)。亚组分析证实了模型在不同年龄、性别和合并症亚组中的稳健表现。SHAP分析确定尿量、机械通气、序期器官衰竭评估评分、肌酐、格拉斯哥昏迷量表评分和肾毒性药物使用为核心预测特征。决策曲线分析证实,LightGBM在不同阈值范围内提供一致的临床获益。最优的LightGBM模型被部署为一个可公开访问的基于web的预测应用程序,具有集成的SHAP可解释性。结论:本研究开发并验证了一个动态、分层的预测系统,该系统可为中重度SA-AKI提供特定阶段的风险评估。该系统经过了多地区、多中心的严格验证,并转化为可解释的临床决策支持工具,为精准管理提供了科学依据。
期刊介绍:
The Journal of Medical Internet Research (JMIR) is a highly respected publication in the field of health informatics and health services. With a founding date in 1999, JMIR has been a pioneer in the field for over two decades.
As a leader in the industry, the journal focuses on digital health, data science, health informatics, and emerging technologies for health, medicine, and biomedical research. It is recognized as a top publication in these disciplines, ranking in the first quartile (Q1) by Impact Factor.
Notably, JMIR holds the prestigious position of being ranked #1 on Google Scholar within the "Medical Informatics" discipline.