External validation of three scores for predicting prehospital return of spontaneous circulation in out-of-hospital cardiac arrest

Abstract

Background

Although three established models for predicting the return of spontaneous circulation (ROSC) in out-of-hospital cardiac arrest (OHCA) exist, combinational external validation of these models remains limited. This study aimed to externally validate and compare the performance of three predictive models—RACA, P-ROSC, and UB-ROSC–and provide evidence to guide the selection and application of predictive models for prehospital ROSC in diverse settings.

Methods

A retrospective validation was conducted using the National Taiwan University Hospital Hsinchu and Yunlin Branch Out-of-Hospital Cardiac Arrest Research Databases. Patients with EMS-treated OHCAs admitted to the hospital between January 2016 and July 2023 were recruited. The primary outcome was prehospital ROSC. Model performance was evaluated using discrimination, calibration, sensitivity, specificity, positive predictive value, negative predictive value, and diagnostic odds ratio. Calibration and density distribution plots were generated.

Results

All three models demonstrated moderate-to-high discrimination with AUROCs of 0.758 (RACA), 0.755 (P-ROSC), and 0.747 (UB-ROSC). The RACA score exhibited better calibration across the risk deciles, whereas the P-ROSC and UB-ROSC scores tended to overestimate the probabilities at higher predicted risk levels. The P-ROSC score required fewer variables and showed the best separation between prehospital and non-prehospital ROSC cases. Optimal cut-off values for the RACA, P-ROSC, and UB-ROSC scores were 0.45, 41, and − 13, respectively, with corresponding sensitivities of 62 %, 56 %, and 71 % and specificities of 78 %, 82 %, and 69 %. All models achieved high NPVs (>96 %), but PPVs remained low (16–21 %).

Conclusions

The P-ROSC, which requires fewer variables, has emerged as the most practical model for Taiwanese populations. However, the choice of the model should be guided by the availability of variables, regional EMS characteristics, and trends in prehospital ROSC rates.