Sriram Ramgopal, Melissa Neveu, Douglas Lorenz, Jillian Benedetti, Jack Lavey, Todd A Florin
{"title":"两种小儿肺炎临床预测模型的外部验证。","authors":"Sriram Ramgopal, Melissa Neveu, Douglas Lorenz, Jillian Benedetti, Jack Lavey, Todd A Florin","doi":"10.1016/j.acap.2024.08.009","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>To externally validate two prediction models for pediatric radiographic pneumonia.</p><p><strong>Methods: </strong>We prospectively evaluated the performance of two prediction models (Pneumonia Risk Score [PRS] and CARPE DIEM models) from a prospective convenience sample of children 90 days - 18 years of age from a pediatric emergency department undergoing chest radiography for suspected pneumonia between January 1, 2022, to December 31<sup>st</sup>, 2023. We evaluated model performance using the original intercepts and coefficients and evaluated for performance changes when performing recalibration and re-estimation procedures.</p><p><strong>Results: </strong>We included 202 patients (median age 3 years, IQR 1-6 years), of whom radiographic pneumonia was found in 92 (41.0%). The PRS model had an area under the receiver operator characteristic curve of 0.72 (95% confidence interval [CI] 0.64-0.79), which was higher than the CARPE DIEM (0.59; 95% CI 0.51-0.67) (P<0.01). Using optimal cutpoints, the PRS model showed higher sensitivity (65.2%, 95% CI 54.6-74.9) and specificity (72.7%, 95% CI 63.4-80.8) compared to the CARPE DIEM model (sensitivity 56.5 [95% CI 45.8-66.8]; specificity 60.9 [95% CI 50.2-69.2]). Recalibration and re-estimation of models improved performance, particularly for the CARPE DIEM model, with gains in sensitivity and specificity, and improved calibration.</p><p><strong>Conclusion: </strong>The PRS model demonstrated better performance than the CARPE DIEM model in predicting radiographic pneumonia. Among children with a high rate of pneumonia, these models did not reach a level of performance sufficient to be used independently of clinical judgement. These findings highlight the need for further validation and improvement of models to enhance their utility.</p>","PeriodicalId":50930,"journal":{"name":"Academic Pediatrics","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2024-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"External validation of two clinical prediction models for pediatric pneumonia.\",\"authors\":\"Sriram Ramgopal, Melissa Neveu, Douglas Lorenz, Jillian Benedetti, Jack Lavey, Todd A Florin\",\"doi\":\"10.1016/j.acap.2024.08.009\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Objective: </strong>To externally validate two prediction models for pediatric radiographic pneumonia.</p><p><strong>Methods: </strong>We prospectively evaluated the performance of two prediction models (Pneumonia Risk Score [PRS] and CARPE DIEM models) from a prospective convenience sample of children 90 days - 18 years of age from a pediatric emergency department undergoing chest radiography for suspected pneumonia between January 1, 2022, to December 31<sup>st</sup>, 2023. We evaluated model performance using the original intercepts and coefficients and evaluated for performance changes when performing recalibration and re-estimation procedures.</p><p><strong>Results: </strong>We included 202 patients (median age 3 years, IQR 1-6 years), of whom radiographic pneumonia was found in 92 (41.0%). The PRS model had an area under the receiver operator characteristic curve of 0.72 (95% confidence interval [CI] 0.64-0.79), which was higher than the CARPE DIEM (0.59; 95% CI 0.51-0.67) (P<0.01). Using optimal cutpoints, the PRS model showed higher sensitivity (65.2%, 95% CI 54.6-74.9) and specificity (72.7%, 95% CI 63.4-80.8) compared to the CARPE DIEM model (sensitivity 56.5 [95% CI 45.8-66.8]; specificity 60.9 [95% CI 50.2-69.2]). Recalibration and re-estimation of models improved performance, particularly for the CARPE DIEM model, with gains in sensitivity and specificity, and improved calibration.</p><p><strong>Conclusion: </strong>The PRS model demonstrated better performance than the CARPE DIEM model in predicting radiographic pneumonia. Among children with a high rate of pneumonia, these models did not reach a level of performance sufficient to be used independently of clinical judgement. These findings highlight the need for further validation and improvement of models to enhance their utility.</p>\",\"PeriodicalId\":50930,\"journal\":{\"name\":\"Academic Pediatrics\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.0000,\"publicationDate\":\"2024-08-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Academic Pediatrics\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1016/j.acap.2024.08.009\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PEDIATRICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Academic Pediatrics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.acap.2024.08.009","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PEDIATRICS","Score":null,"Total":0}
External validation of two clinical prediction models for pediatric pneumonia.
Objective: To externally validate two prediction models for pediatric radiographic pneumonia.
Methods: We prospectively evaluated the performance of two prediction models (Pneumonia Risk Score [PRS] and CARPE DIEM models) from a prospective convenience sample of children 90 days - 18 years of age from a pediatric emergency department undergoing chest radiography for suspected pneumonia between January 1, 2022, to December 31st, 2023. We evaluated model performance using the original intercepts and coefficients and evaluated for performance changes when performing recalibration and re-estimation procedures.
Results: We included 202 patients (median age 3 years, IQR 1-6 years), of whom radiographic pneumonia was found in 92 (41.0%). The PRS model had an area under the receiver operator characteristic curve of 0.72 (95% confidence interval [CI] 0.64-0.79), which was higher than the CARPE DIEM (0.59; 95% CI 0.51-0.67) (P<0.01). Using optimal cutpoints, the PRS model showed higher sensitivity (65.2%, 95% CI 54.6-74.9) and specificity (72.7%, 95% CI 63.4-80.8) compared to the CARPE DIEM model (sensitivity 56.5 [95% CI 45.8-66.8]; specificity 60.9 [95% CI 50.2-69.2]). Recalibration and re-estimation of models improved performance, particularly for the CARPE DIEM model, with gains in sensitivity and specificity, and improved calibration.
Conclusion: The PRS model demonstrated better performance than the CARPE DIEM model in predicting radiographic pneumonia. Among children with a high rate of pneumonia, these models did not reach a level of performance sufficient to be used independently of clinical judgement. These findings highlight the need for further validation and improvement of models to enhance their utility.
期刊介绍:
Academic Pediatrics, the official journal of the Academic Pediatric Association, is a peer-reviewed publication whose purpose is to strengthen the research and educational base of academic general pediatrics. The journal provides leadership in pediatric education, research, patient care and advocacy. Content areas include pediatric education, emergency medicine, injury, abuse, behavioral pediatrics, holistic medicine, child health services and health policy,and the environment. The journal provides an active forum for the presentation of pediatric educational research in diverse settings, involving medical students, residents, fellows, and practicing professionals. The journal also emphasizes important research relating to the quality of child health care, health care policy, and the organization of child health services. It also includes systematic reviews of primary care interventions and important methodologic papers to aid research in child health and education.