Erica Frechman, Byron C Jaeger, Marc Kowalkowski, Jeff D Williamson, Kristin M Lenoir, Jessica A Palakshappa, Brian J Wells, Kathryn E Callahan, Nicholas M Pajewski, Jennifer L Gabbard
{"title":"External validation of a proprietary risk model for 1-year mortality in community-dwelling adults aged 65 years or older.","authors":"Erica Frechman, Byron C Jaeger, Marc Kowalkowski, Jeff D Williamson, Kristin M Lenoir, Jessica A Palakshappa, Brian J Wells, Kathryn E Callahan, Nicholas M Pajewski, Jennifer L Gabbard","doi":"10.1093/jamia/ocaf062","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>To examine the discrimination, calibration, and algorithmic fairness of the Epic End of Life Care Index (EOL-CI).</p><p><strong>Materials and methods: </strong>We assessed the EOL-CI's performance by estimating area under the receiver operating characteristic curve (AUC), sensitivity, and positive and negative predictive values in community-dwelling adults ≥65 years of age in a single health system in the Southeastern United States. Algorithmic fairness was examined by comparing the model's performance across sex, race, and ethnicity subgroups. Using a machine learning approach, we also explored local re-calibration of the EOL-CI considering additional information on past hospitalizations and frailty.</p><p><strong>Results: </strong>Among 215 731 patients (median age = 74 years, 57% female, 12% of Black race), 10% were classified as medium risk (15-44) and 3% as high risk (≥45) by the EOL-CI. The observed 1-year mortality rate was 3%. The EOL-CI had an AUC 0.82 for 1-year mortality, with a positive predictive value of 22%. Predictive performance was generally similar across sex and race subgroups, though the EOL-CI displayed better performance with increasing age and in older adults with 2 or more outpatient encounters in the past 24 months. Local re-calibration of the EOL-CI was required to provide absolute estimates of mortality risk, and calibration was further improved when the EOL-CI was augmented with data on inpatient hospitalizations and frailty.</p><p><strong>Discussion: </strong>The EOL-CI demonstrates reasonable discrimination, albeit with better performance in older adults and in those with greater health system contact.</p><p><strong>Conclusion: </strong>Local refinement and calibration of the EOL-CI score is required to provide direct estimates of prognosis, with the goal of making the EOL-CI a more a valuable tool at the point of care for identifying patients who would benefit from targeted palliative care interventions and proactive care planning.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1110-1119"},"PeriodicalIF":4.7000,"publicationDate":"2025-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12199354/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the American Medical Informatics Association","FirstCategoryId":"91","ListUrlMain":"https://doi.org/10.1093/jamia/ocaf062","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Objective: To examine the discrimination, calibration, and algorithmic fairness of the Epic End of Life Care Index (EOL-CI).
Materials and methods: We assessed the EOL-CI's performance by estimating area under the receiver operating characteristic curve (AUC), sensitivity, and positive and negative predictive values in community-dwelling adults ≥65 years of age in a single health system in the Southeastern United States. Algorithmic fairness was examined by comparing the model's performance across sex, race, and ethnicity subgroups. Using a machine learning approach, we also explored local re-calibration of the EOL-CI considering additional information on past hospitalizations and frailty.
Results: Among 215 731 patients (median age = 74 years, 57% female, 12% of Black race), 10% were classified as medium risk (15-44) and 3% as high risk (≥45) by the EOL-CI. The observed 1-year mortality rate was 3%. The EOL-CI had an AUC 0.82 for 1-year mortality, with a positive predictive value of 22%. Predictive performance was generally similar across sex and race subgroups, though the EOL-CI displayed better performance with increasing age and in older adults with 2 or more outpatient encounters in the past 24 months. Local re-calibration of the EOL-CI was required to provide absolute estimates of mortality risk, and calibration was further improved when the EOL-CI was augmented with data on inpatient hospitalizations and frailty.
Discussion: The EOL-CI demonstrates reasonable discrimination, albeit with better performance in older adults and in those with greater health system contact.
Conclusion: Local refinement and calibration of the EOL-CI score is required to provide direct estimates of prognosis, with the goal of making the EOL-CI a more a valuable tool at the point of care for identifying patients who would benefit from targeted palliative care interventions and proactive care planning.
期刊介绍：
JAMIA is AMIA''s premier peer-reviewed journal for biomedical and health informatics. Covering the full spectrum of activities in the field, JAMIA includes informatics articles in the areas of clinical care, clinical research, translational science, implementation science, imaging, education, consumer health, public health, and policy. JAMIA''s articles describe innovative informatics research and systems that help to advance biomedical science and to promote health. Case reports, perspectives and reviews also help readers stay connected with the most important informatics developments in implementation, policy and education.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
