External Validation of Eight Ruptured Abdominal Aortic Aneurysm Mortality Predictive Models Demonstrates Limited Predictive Accuracy.

Abstract

Objective: Over a dozen ruptured abdominal aortic aneurysm (rAAA) mortality risk prediction models currently exist; however, lack of external validation limits their applicability. This study aimed to evaluate the accuracy of eight common rAAA mortality risk prediction models in a large, contemporary, external validation cohort.

Methods: A retrospective review of rAAA repairs at a multicentre integrated regional healthcare system with large central quaternary referral facility (2010 - 2020) was performed. Eight models were used to predict 30 day post-operative death, including the Updated Glasgow Aneurysm Score (GAS), Vascular Study Group of New England rAAA Risk Score, Harborview Preoperative rAAA Risk Score, Modified Harborview Risk Score, Vancouver Scoring System (VSS), Artificial Neural Network Score, Dutch Aneurysm Score, and Edinburgh Ruptured Aneurysm Score. The models were assessed for discrimination, calibration, and clinical utility using receiver operating characteristic curves (area under the curve [AUC]), Hosmer-Lemeshow χ² test, Brier scores, and decision curve analysis. The proportion of unexpected survivors (survival despite > 80% predicted 30 day mortality) to expected deceased was compared across calculators, and both groups were compared using the model demonstrating the highest unexpected survival frequency.

Results: A total of 315 rAAA repairs were included (mean age 73.6 ± 10.0 years; 72.1% male; 49.8% open repair) with a 30 day mortality rate of 32.1%. Three models had fair discrimination (AUC ≥ 0.70), with GAS having the highest AUC (0.74, 95% confidence interval 0.68 - 0.79). All models demonstrated poor to adequate calibration. Using VSS, unexpected survivors (n= 25) had less pre-operative shock (72% vs. 96%; p=.050) and statistically significantly less coagulopathy (median international normalised ratio 1.2 [interquartile range 1.1, 1.5] vs. 1.8 [1.3, 2.2]; p= .015) compared with expected deceased (n=23).

Conclusion: Current rAAA risk prediction models demonstrated fair discrimination and poor to adequate calibration. These findings suggest that existing risk prediction models have not sufficiently captured important physiological characteristics associated with rAAA mortality and should be cautiously applied to clinical practice.