External Validation of a Machine Learning Model for the Early Prediction of Upper-Limb Recovery After Stroke.

Abstract

Background: Reliable prediction of upper-limb recovery after stroke can support rehabilitation planning, yet few prognostic models have been tested beyond their development cohorts. External validation is essential to establish the clinical reliability of these methods.

Objective: To externally validate a previously developed machine learning model for predicting 6-month upper-limb capacity after stroke, measured by the Action Research Arm Test (ARAT).

Methods: The model was developed on a multicentre Dutch cohort of first-ever ischaemic stroke patients and validated in an independent Danish prospective cohort, including both ischaemic and haemorrhagic strokes. Predictions were generated from baseline assessments at 2 weeks post-stroke. Model performance was evaluated using the median absolute error (MedAE) and calibration analysis.

Results: The validation cohort comprised 80 patients assessed at 14 ± 4 days post-stroke. Overall prediction error was comparable between the validation and development cohorts (MedAE = 5.7 [IQR 1.9-15.1] vs 3.9 [IQR 1.1-13.0]; P = 0.12). Calibration was close to ideal for predicted ARAT scores above 40. In contrast, lower predicted ranges showed systematic underprediction, reflecting variable outcomes in severely impaired patients, a pattern similar to that observed in the development cohort.

Conclusions: In an independent validation cohort, the machine learning model performed similarly to its development cohort but showed clinically relevant miscalibration in patients with low predicted ARAT scores. Inclusion of additional predictors is required to improve reliability in severely impaired patients before subsequent steps toward clinical implementation can be considered.