Nassr Nama, Ye Shen, Jeffrey N Bone, Zerlyn Lee, Kara Picco, Falla Jin, Jessica L Foulds, Josée Anne Gagnon, Chris Novak, Brigitte Parisien, Matthew Donlan, Ran D Goldman, Anupam Sehgal, Joanna Holland, Sanjay Mahant, Joel S Tieder, Peter J Gill
{"title":"针对严重基础诊断的简短已解决不明原因事件预测规则的外部验证。","authors":"Nassr Nama, Ye Shen, Jeffrey N Bone, Zerlyn Lee, Kara Picco, Falla Jin, Jessica L Foulds, Josée Anne Gagnon, Chris Novak, Brigitte Parisien, Matthew Donlan, Ran D Goldman, Anupam Sehgal, Joanna Holland, Sanjay Mahant, Joel S Tieder, Peter J Gill","doi":"10.1001/jamapediatrics.2024.4399","DOIUrl":null,"url":null,"abstract":"<p><strong>Importance: </strong>The American Academy of Pediatrics (AAP) higher-risk criteria for brief resolved unexplained events (BRUE) have a low positive predictive value (4.8%) and misclassify most infants as higher risk (>90%). New BRUE prediction rules from a US cohort of 3283 infants showed improved discrimination; however, these rules have not been validated in an external cohort.</p><p><strong>Objective: </strong>To externally validate new BRUE prediction rules and compare them with the AAP higher-risk criteria.</p><p><strong>Design, setting, and participants: </strong>This was a retrospective multicenter cohort study conducted from 2017 to 2021 and monitored for 90 days after index presentation. The setting included infants younger than 1 year with a BRUE identified through retrospective chart review from 11 Canadian hospitals. Study data were analyzed from March 2022 to March 2024.</p><p><strong>Exposures: </strong>The BRUE prediction rules.</p><p><strong>Main outcome and measure: </strong>The primary outcome was a serious underlying diagnosis, defined as conditions where a delay in diagnosis could lead to increased morbidity or mortality.</p><p><strong>Results: </strong>Of 1042 patients (median [IQR] age, 41 [13-84] days; 529 female [50.8%]), 977 (93.8%) were classified as higher risk by the AAP criteria. A total of 79 patients (7.6%) had a serious underlying diagnosis. For this outcome, the AAP criteria demonstrated a sensitivity of 100.0% (95% CI, 95.4%-100.0%), a specificity of 6.7% (95% CI, 5.2%-8.5%), a positive likelihood ratio (LR+) of 1.07 (95% CI, 1.05-1.09), and an AUC of 0.53 (95% CI, 0.53-0.54). The BRUE prediction rule for discerning serious diagnoses displayed an AUC of 0.60 (95% CI, 0.54-0.67; calibration intercept: 0.60), which improved to an AUC of 0.71 (95% CI, 0.65-0.76; P < .001; calibration intercept: 0.00) after model revision. Event recurrence was noted in 163 patients (15.6%). For this outcome, the AAP criteria yielded a sensitivity of 99.4% (95% CI, 96.6%-100.0%), a specificity of 7.3% (95% CI, 5.7%-9.2%), an LR+ of 1.07 (95% CI, 1.05-1.10), and an AUC of 0.58 (95% CI, 0.56-0.58). The AUC of the prediction rule stood at 0.67 (95% CI, 0.62-0.72; calibration intercept: 0.15).</p><p><strong>Conclusions and relevance: </strong>Results of this multicenter cohort study show that the BRUE prediction rules outperformed the AAP higher-risk criteria on external geographical validation, and performance improved after recalibration. These rules provide clinicians and families with a more precise tool to support decision-making, grounded in individual risk tolerance.</p>","PeriodicalId":14683,"journal":{"name":"JAMA Pediatrics","volume":" ","pages":""},"PeriodicalIF":24.7000,"publicationDate":"2024-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"External Validation of Brief Resolved Unexplained Events Prediction Rules for Serious Underlying Diagnosis.\",\"authors\":\"Nassr Nama, Ye Shen, Jeffrey N Bone, Zerlyn Lee, Kara Picco, Falla Jin, Jessica L Foulds, Josée Anne Gagnon, Chris Novak, Brigitte Parisien, Matthew Donlan, Ran D Goldman, Anupam Sehgal, Joanna Holland, Sanjay Mahant, Joel S Tieder, Peter J Gill\",\"doi\":\"10.1001/jamapediatrics.2024.4399\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Importance: </strong>The American Academy of Pediatrics (AAP) higher-risk criteria for brief resolved unexplained events (BRUE) have a low positive predictive value (4.8%) and misclassify most infants as higher risk (>90%). New BRUE prediction rules from a US cohort of 3283 infants showed improved discrimination; however, these rules have not been validated in an external cohort.</p><p><strong>Objective: </strong>To externally validate new BRUE prediction rules and compare them with the AAP higher-risk criteria.</p><p><strong>Design, setting, and participants: </strong>This was a retrospective multicenter cohort study conducted from 2017 to 2021 and monitored for 90 days after index presentation. The setting included infants younger than 1 year with a BRUE identified through retrospective chart review from 11 Canadian hospitals. Study data were analyzed from March 2022 to March 2024.</p><p><strong>Exposures: </strong>The BRUE prediction rules.</p><p><strong>Main outcome and measure: </strong>The primary outcome was a serious underlying diagnosis, defined as conditions where a delay in diagnosis could lead to increased morbidity or mortality.</p><p><strong>Results: </strong>Of 1042 patients (median [IQR] age, 41 [13-84] days; 529 female [50.8%]), 977 (93.8%) were classified as higher risk by the AAP criteria. A total of 79 patients (7.6%) had a serious underlying diagnosis. For this outcome, the AAP criteria demonstrated a sensitivity of 100.0% (95% CI, 95.4%-100.0%), a specificity of 6.7% (95% CI, 5.2%-8.5%), a positive likelihood ratio (LR+) of 1.07 (95% CI, 1.05-1.09), and an AUC of 0.53 (95% CI, 0.53-0.54). The BRUE prediction rule for discerning serious diagnoses displayed an AUC of 0.60 (95% CI, 0.54-0.67; calibration intercept: 0.60), which improved to an AUC of 0.71 (95% CI, 0.65-0.76; P < .001; calibration intercept: 0.00) after model revision. Event recurrence was noted in 163 patients (15.6%). For this outcome, the AAP criteria yielded a sensitivity of 99.4% (95% CI, 96.6%-100.0%), a specificity of 7.3% (95% CI, 5.7%-9.2%), an LR+ of 1.07 (95% CI, 1.05-1.10), and an AUC of 0.58 (95% CI, 0.56-0.58). The AUC of the prediction rule stood at 0.67 (95% CI, 0.62-0.72; calibration intercept: 0.15).</p><p><strong>Conclusions and relevance: </strong>Results of this multicenter cohort study show that the BRUE prediction rules outperformed the AAP higher-risk criteria on external geographical validation, and performance improved after recalibration. These rules provide clinicians and families with a more precise tool to support decision-making, grounded in individual risk tolerance.</p>\",\"PeriodicalId\":14683,\"journal\":{\"name\":\"JAMA Pediatrics\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":24.7000,\"publicationDate\":\"2024-12-16\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"JAMA Pediatrics\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1001/jamapediatrics.2024.4399\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PEDIATRICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"JAMA Pediatrics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1001/jamapediatrics.2024.4399","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PEDIATRICS","Score":null,"Total":0}
External Validation of Brief Resolved Unexplained Events Prediction Rules for Serious Underlying Diagnosis.
Importance: The American Academy of Pediatrics (AAP) higher-risk criteria for brief resolved unexplained events (BRUE) have a low positive predictive value (4.8%) and misclassify most infants as higher risk (>90%). New BRUE prediction rules from a US cohort of 3283 infants showed improved discrimination; however, these rules have not been validated in an external cohort.
Objective: To externally validate new BRUE prediction rules and compare them with the AAP higher-risk criteria.
Design, setting, and participants: This was a retrospective multicenter cohort study conducted from 2017 to 2021 and monitored for 90 days after index presentation. The setting included infants younger than 1 year with a BRUE identified through retrospective chart review from 11 Canadian hospitals. Study data were analyzed from March 2022 to March 2024.
Exposures: The BRUE prediction rules.
Main outcome and measure: The primary outcome was a serious underlying diagnosis, defined as conditions where a delay in diagnosis could lead to increased morbidity or mortality.
Results: Of 1042 patients (median [IQR] age, 41 [13-84] days; 529 female [50.8%]), 977 (93.8%) were classified as higher risk by the AAP criteria. A total of 79 patients (7.6%) had a serious underlying diagnosis. For this outcome, the AAP criteria demonstrated a sensitivity of 100.0% (95% CI, 95.4%-100.0%), a specificity of 6.7% (95% CI, 5.2%-8.5%), a positive likelihood ratio (LR+) of 1.07 (95% CI, 1.05-1.09), and an AUC of 0.53 (95% CI, 0.53-0.54). The BRUE prediction rule for discerning serious diagnoses displayed an AUC of 0.60 (95% CI, 0.54-0.67; calibration intercept: 0.60), which improved to an AUC of 0.71 (95% CI, 0.65-0.76; P < .001; calibration intercept: 0.00) after model revision. Event recurrence was noted in 163 patients (15.6%). For this outcome, the AAP criteria yielded a sensitivity of 99.4% (95% CI, 96.6%-100.0%), a specificity of 7.3% (95% CI, 5.7%-9.2%), an LR+ of 1.07 (95% CI, 1.05-1.10), and an AUC of 0.58 (95% CI, 0.56-0.58). The AUC of the prediction rule stood at 0.67 (95% CI, 0.62-0.72; calibration intercept: 0.15).
Conclusions and relevance: Results of this multicenter cohort study show that the BRUE prediction rules outperformed the AAP higher-risk criteria on external geographical validation, and performance improved after recalibration. These rules provide clinicians and families with a more precise tool to support decision-making, grounded in individual risk tolerance.
期刊介绍:
JAMA Pediatrics, the oldest continuously published pediatric journal in the US since 1911, is an international peer-reviewed publication and a part of the JAMA Network. Published weekly online and in 12 issues annually, it garners over 8.4 million article views and downloads yearly. All research articles become freely accessible online after 12 months without any author fees, and through the WHO's HINARI program, the online version is accessible to institutions in developing countries.
With a focus on advancing the health of infants, children, and adolescents, JAMA Pediatrics serves as a platform for discussing crucial issues and policies in child and adolescent health care. Leveraging the latest technology, it ensures timely access to information for its readers worldwide.