External validation of two clinical prediction models for pediatric pneumonia.

Abstract

Objective: To externally validate two prediction models for pediatric radiographic pneumonia.

Methods: We prospectively evaluated the performance of two prediction models (Pneumonia Risk Score [PRS] and CARPE DIEM models) from a prospective convenience sample of children 90 days - 18 years of age from a pediatric emergency department undergoing chest radiography for suspected pneumonia between January 1, 2022, to December 31^st, 2023. We evaluated model performance using the original intercepts and coefficients and evaluated for performance changes when performing recalibration and re-estimation procedures.

Results: We included 202 patients (median age 3 years, IQR 1-6 years), of whom radiographic pneumonia was found in 92 (41.0%). The PRS model had an area under the receiver operator characteristic curve of 0.72 (95% confidence interval [CI] 0.64-0.79), which was higher than the CARPE DIEM (0.59; 95% CI 0.51-0.67) (P<0.01). Using optimal cutpoints, the PRS model showed higher sensitivity (65.2%, 95% CI 54.6-74.9) and specificity (72.7%, 95% CI 63.4-80.8) compared to the CARPE DIEM model (sensitivity 56.5 [95% CI 45.8-66.8]; specificity 60.9 [95% CI 50.2-69.2]). Recalibration and re-estimation of models improved performance, particularly for the CARPE DIEM model, with gains in sensitivity and specificity, and improved calibration.

Conclusion: The PRS model demonstrated better performance than the CARPE DIEM model in predicting radiographic pneumonia. Among children with a high rate of pneumonia, these models did not reach a level of performance sufficient to be used independently of clinical judgement. These findings highlight the need for further validation and improvement of models to enhance their utility.