Ben Li, Badr Aljabri, Derek Beaton, Mohamad A Hussain, Douglas S Lee, Duminda N Wijeysundera, Ori D Rotstein, Charles de Mestral, Muhammad Mamdani, Graham Roche-Nagle, Mohammed Al-Omran
{"title":"Predicting Outcomes Following Carotid Artery Stenting Using Machine Learning.","authors":"Ben Li, Badr Aljabri, Derek Beaton, Mohamad A Hussain, Douglas S Lee, Duminda N Wijeysundera, Ori D Rotstein, Charles de Mestral, Muhammad Mamdani, Graham Roche-Nagle, Mohammed Al-Omran","doi":"10.1177/15266028251333670","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Carotid artery stenting (CAS) carries important perioperative risks. Outcome prediction tools may help guide clinical decision-making but remain limited. We developed machine learning (ML) algorithms that predict 30-day outcomes following transfemoral CAS.</p><p><strong>Methods: </strong>The National Surgical Quality Improvement Program (NSQIP) targeted vascular database was used to identify patients who underwent transfemoral CAS between 2011 and 2021. Input features included 36 preoperative demographic/clinical variables. The primary outcome was a 30-day major adverse cardiovascular event (MACE; composite of stroke, myocardial infarction [MI], or death). The secondary outcomes were 30-day stroke, MI, death, carotid-related morbidity, other morbidity, non-home discharge, and unplanned readmission. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features with logistic regression as the baseline comparator. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. Variable importance scores were calculated to determine the top 10 predictive features. Performance was assessed on subgroups based on age, sex, race, ethnicity, symptom status, stent type, and urgency.</p><p><strong>Results: </strong>Overall, 2093 patients underwent CAS during the study period. Thirty-day MACE occurred in 130 (6.2%) patients. The best-performing prediction model for 30-day MACE was XGBoost, achieving an AUROC (95% CI) of 0.93 (0.92-0.94). In comparison, logistic regression had an AUROC (95% CI) of 0.67 (0.65-0.68), and existing tools in the literature demonstrate AUROCs ranging from 0.58 to 0.74. For secondary outcomes, XGBoost achieved AUROCs between 0.86 and 0.97. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.02. The top three predictive features in our algorithm were (1) symptomatic carotid stenosis, (2) age, and (3) American Society of Anesthesiologists classification. Model performance remained robust on all subgroup analyses of specific demographic and clinical populations.</p><p><strong>Conclusions: </strong>Our ML models accurately predict 30-day outcomes following transfemoral CAS using preoperative data. They have the potential for important utility in guiding risk-mitigation strategies for patients being considered for CAS to improve outcomes.Clinical ImpactTransfemoral carotid artery stenting (CAS) carries important perioperative risks. Outcome prediction tools may help guide clinical decision-making but remain limited. Using data from the National Surgical Quality Improvement Program (NSQIP) targeted vascular database, we developed machine learning (ML) models that accurately predict 30-day outcomes following transfemoral CAS using preoperative data, outperforming logistic regression and existing tools in the literature. The models were well-calibrated and remained robust across demographic and clinical subpopulations. These ML algorithms have the potential for important utility in guiding risk-mitigation strategies for patients being considered for transfemoral CAS to improve outcomes.</p>","PeriodicalId":50210,"journal":{"name":"Journal of Endovascular Therapy","volume":" ","pages":"15266028251333670"},"PeriodicalIF":1.7000,"publicationDate":"2025-04-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Endovascular Therapy","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1177/15266028251333670","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"PERIPHERAL VASCULAR DISEASE","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Carotid artery stenting (CAS) carries important perioperative risks. Outcome prediction tools may help guide clinical decision-making but remain limited. We developed machine learning (ML) algorithms that predict 30-day outcomes following transfemoral CAS.
Methods: The National Surgical Quality Improvement Program (NSQIP) targeted vascular database was used to identify patients who underwent transfemoral CAS between 2011 and 2021. Input features included 36 preoperative demographic/clinical variables. The primary outcome was a 30-day major adverse cardiovascular event (MACE; composite of stroke, myocardial infarction [MI], or death). The secondary outcomes were 30-day stroke, MI, death, carotid-related morbidity, other morbidity, non-home discharge, and unplanned readmission. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features with logistic regression as the baseline comparator. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. Variable importance scores were calculated to determine the top 10 predictive features. Performance was assessed on subgroups based on age, sex, race, ethnicity, symptom status, stent type, and urgency.
Results: Overall, 2093 patients underwent CAS during the study period. Thirty-day MACE occurred in 130 (6.2%) patients. The best-performing prediction model for 30-day MACE was XGBoost, achieving an AUROC (95% CI) of 0.93 (0.92-0.94). In comparison, logistic regression had an AUROC (95% CI) of 0.67 (0.65-0.68), and existing tools in the literature demonstrate AUROCs ranging from 0.58 to 0.74. For secondary outcomes, XGBoost achieved AUROCs between 0.86 and 0.97. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.02. The top three predictive features in our algorithm were (1) symptomatic carotid stenosis, (2) age, and (3) American Society of Anesthesiologists classification. Model performance remained robust on all subgroup analyses of specific demographic and clinical populations.
Conclusions: Our ML models accurately predict 30-day outcomes following transfemoral CAS using preoperative data. They have the potential for important utility in guiding risk-mitigation strategies for patients being considered for CAS to improve outcomes.Clinical ImpactTransfemoral carotid artery stenting (CAS) carries important perioperative risks. Outcome prediction tools may help guide clinical decision-making but remain limited. Using data from the National Surgical Quality Improvement Program (NSQIP) targeted vascular database, we developed machine learning (ML) models that accurately predict 30-day outcomes following transfemoral CAS using preoperative data, outperforming logistic regression and existing tools in the literature. The models were well-calibrated and remained robust across demographic and clinical subpopulations. These ML algorithms have the potential for important utility in guiding risk-mitigation strategies for patients being considered for transfemoral CAS to improve outcomes.
期刊介绍:
The Journal of Endovascular Therapy (formerly the Journal of Endovascular Surgery) was established in 1994 as a forum for all physicians, scientists, and allied healthcare professionals who are engaged or interested in peripheral endovascular techniques and technology. An official publication of the International Society of Endovascular Specialists (ISEVS), the Journal of Endovascular Therapy publishes peer-reviewed articles of interest to clinicians and researchers in the field of peripheral endovascular interventions.