Development and internal validation of machine learning-based models and external validation of existing risk scores for outcome prediction in patients with ischemic stroke

We developed new ML models and externally validated existing statistical models (ischemic stroke predictive risk score [iScore] and totaled health risks in vascular events [THRIVE] scores) for predicting the composite of recurrent stroke or all-cause mortality at 90 days and at 3 years after hospitalization for first AIS. In adults hospitalized with AIS from January 2005 to November 2016, with follow-up until November 2019, we developed three ML models (random forest [RF], support vector machine [SVM], and extreme gradient boosting [XGBOOST]) and externally validated the iScore and THRIVE scores for predicting the composite outcomes after AIS hospitalization, using data from 721 patients and 90 potential predictor variables. At 90 days and 3 years, 11% and 34% of patients, respectively, reached the composite outcome. For the 90-day prediction, the area under the receiver operating characteristic curves (AUC) was 0.779 for RF, 0.771 for SVM, 0.772 for XGBOOST, 0.720 for iScore, and 0.664 for THRIVE. For 3-year prediction the AUC was 0.743 for RF, 0.777 for SVM, 0.773 for XGBOOST, 0.710 for iScore, and 0.675 for THRIVE. The study provided three ML-based predictive models that achieved good discrimination and clinical usefulness in outcome prediction after AIS and broadened the application of the iScore and THRIVE scoring system for long-term outcome prediction. Our findings warrant comparative analyses of ML and existing statistical method-based risk prediction tools for outcome prediction after AIS in new datasets.