Development and Evaluation of an Electronic Health Record-Generated Clinical Coverage Scoring System Compared to Human Decision-Making in Pediatric Surgical Patients: A Single Center Experience.
Pornswan Ngamprasertwong, Annie R Amin, Jiwon Lee, Lili Ding, Bobby R Das, Ali I Kandil, Theerawich Likitabhorn, Michelle Coleman, Simran Venkatraman, Sarah Wilhelm, Ekanong Sutthipongkiat, Veronica O Busso
{"title":"Development and Evaluation of an Electronic Health Record-Generated Clinical Coverage Scoring System Compared to Human Decision-Making in Pediatric Surgical Patients: A Single Center Experience.","authors":"Pornswan Ngamprasertwong, Annie R Amin, Jiwon Lee, Lili Ding, Bobby R Das, Ali I Kandil, Theerawich Likitabhorn, Michelle Coleman, Simran Venkatraman, Sarah Wilhelm, Ekanong Sutthipongkiat, Veronica O Busso","doi":"10.1111/pan.15110","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Surgical Patients in tertiary care centers can be healthy or extremely ill with comorbidities, and procedures vary from simple to difficult and complicated. High-acuity cases, or those involving anesthesia and procedural complexities, require specific anesthesia staff arrangements, specific nursing team assignments, and additional support staff. These cases are identified manually, and there is no ready-to-use scoring system to stratify high-acuity, complex pediatric surgical patients.</p><p><strong>Aims: </strong>We aim to develop an electronic medical record-generated rule-based clinical coverage scoring system to identify high-acuity, complex cases and compare its accuracy with human performance.</p><p><strong>Methods: </strong>In this quality improvement project, an automated scoring system using rule-based clinical criteria was designed and implemented in a quaternary children's hospital. These rules were based on patient characteristics, procedure and anesthetic complexity, and the patient's acute condition. The cases with clinical coverage scores higher than zero were compared to those manually identified as high-acuity, complex cases by the anesthesia clinical directors and operating room charge nurses. The accuracy was reported using sensitivity, specificity, PPV, NPV, accuracy, and F-1 scores.</p><p><strong>Results: </strong>There were 10 761 pediatric surgical cases during the study period (April 7-September 8, 2023). 1450 (13.5%) cases were manually identified as high-acuity, complex cases, while the automated system identified 1906 (17.7%) cases. The accuracy of the automated scoring system improved over time. Eventually, it became better than manual identification with 95.86% (94.48%-97.24%) sensitivity, 99.84% (99.71%-99.98%) specificity, 99.35% (98.78%-99.92%) PPV, 98.97% (98.62%-99.32%) NPV, and 99.04% (98.62%-99.47%) accuracy by the end of the study period. The most impactful interventions were removing canceled cases and adding procedure codes to the rules for automated scores.</p><p><strong>Conclusion: </strong>EHR-generated clinical coverage scores can reliably replace manual reviews of high-acuity, complex pediatric surgical patients. This tool can guide clinical decision-making in real time.</p>","PeriodicalId":19745,"journal":{"name":"Pediatric Anesthesia","volume":" ","pages":"511-519"},"PeriodicalIF":1.7000,"publicationDate":"2025-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pediatric Anesthesia","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1111/pan.15110","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/4/2 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"ANESTHESIOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Surgical Patients in tertiary care centers can be healthy or extremely ill with comorbidities, and procedures vary from simple to difficult and complicated. High-acuity cases, or those involving anesthesia and procedural complexities, require specific anesthesia staff arrangements, specific nursing team assignments, and additional support staff. These cases are identified manually, and there is no ready-to-use scoring system to stratify high-acuity, complex pediatric surgical patients.
Aims: We aim to develop an electronic medical record-generated rule-based clinical coverage scoring system to identify high-acuity, complex cases and compare its accuracy with human performance.
Methods: In this quality improvement project, an automated scoring system using rule-based clinical criteria was designed and implemented in a quaternary children's hospital. These rules were based on patient characteristics, procedure and anesthetic complexity, and the patient's acute condition. The cases with clinical coverage scores higher than zero were compared to those manually identified as high-acuity, complex cases by the anesthesia clinical directors and operating room charge nurses. The accuracy was reported using sensitivity, specificity, PPV, NPV, accuracy, and F-1 scores.
Results: There were 10 761 pediatric surgical cases during the study period (April 7-September 8, 2023). 1450 (13.5%) cases were manually identified as high-acuity, complex cases, while the automated system identified 1906 (17.7%) cases. The accuracy of the automated scoring system improved over time. Eventually, it became better than manual identification with 95.86% (94.48%-97.24%) sensitivity, 99.84% (99.71%-99.98%) specificity, 99.35% (98.78%-99.92%) PPV, 98.97% (98.62%-99.32%) NPV, and 99.04% (98.62%-99.47%) accuracy by the end of the study period. The most impactful interventions were removing canceled cases and adding procedure codes to the rules for automated scores.
Conclusion: EHR-generated clinical coverage scores can reliably replace manual reviews of high-acuity, complex pediatric surgical patients. This tool can guide clinical decision-making in real time.
期刊介绍：
Devoted to the dissemination of research of interest and importance to practising anesthetists everywhere, the scientific and clinical content of Pediatric Anesthesia covers a wide selection of medical disciplines in all areas relevant to paediatric anaesthesia, pain management and peri-operative medicine. The International Editorial Board is supported by the Editorial Advisory Board and a team of Senior Advisors, to ensure that the journal is publishing the best work from the front line of research in the field. The journal publishes high-quality, relevant scientific and clinical research papers, reviews, commentaries, pro-con debates, historical vignettes, correspondence, case presentations and book reviews.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
