Daniel J Schumacher, Benjamin Kinnear, Catherine Michelson, David A Stewart, Bruce E Herman, Adam D Wolfe, Ariel Winn, Jaclyn Boulais, David A Turner, Heather B Howell, Alan Schwartz
{"title":"Development and Validation of a Model to Predict Milestone Levels Based on Entrustable Professional Activity Entrustment-Supervision Levels.","authors":"Daniel J Schumacher, Benjamin Kinnear, Catherine Michelson, David A Stewart, Bruce E Herman, Adam D Wolfe, Ariel Winn, Jaclyn Boulais, David A Turner, Heather B Howell, Alan Schwartz","doi":"10.1097/ACM.0000000000006165","DOIUrl":null,"url":null,"abstract":"<p><strong>Purpose: </strong>Residency programs are increasingly interested in or required to assess residents using Accreditation Council for Graduate Medical Education (ACGME) milestones and specialty-defined entrustable professional activities (EPAs). The authors aimed to develop a model to predict individual residents' milestone levels based on their assigned EPA entrustment-supervision levels.</p><p><strong>Method: </strong>During 3 academic years from 2021 to 2024, the authors conducted a multisite prospective cohort study at 48 U.S. pediatric residency programs. Programs collected entrustment-supervision levels for the 17 general pediatrics EPAs and milestone levels for the 22 ACGME pediatric milestones for every resident biannually. EPA and milestone ratings were assigned by clinical competency committees. The first 4 of 6 biannual data reporting cycles were used to fit multilevel structural equation models and produce equations to generate, for each resident, predicted milestone levels based on EPA entrustment-supervision levels. They developed 2 models: one using 17 EPAs and one using 12 EPAs.</p><p><strong>Results: </strong>Data used for modeling represented 4,328 residents, with 164,886 total entrustment-supervision levels across the general pediatrics EPAs and 243,949 total milestone levels across the pediatric milestones. The fit of the round 1 to 4 model to the round 1 to 4 data (internal prediction) was excellent for both models, with comparative fit indexes of 0.982 (17 EPAs) and 0.981 (12 EPAs). The ability of the round 1 to 4 model to predict milestones for reporting cycles 5 and 6 (external prediction) was similar to the internal predictions, with correlation coefficients of 0.68 (17 EPAs) and 0.69 (12 EPAs) for round 5 and 0.72 (17 EPAs) and 0.68 (12 EPAs) for round 6.</p><p><strong>Conclusions: </strong>This study demonstrates a strong ability to predict milestone levels based on EPA entrustment-supervision levels in a manner that enables meaningful use of EPAs and milestones in assessment efforts at residency programs.</p>","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":""},"PeriodicalIF":5.3000,"publicationDate":"2025-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Academic Medicine","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1097/ACM.0000000000006165","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}
引用次数: 0
Abstract
Purpose: Residency programs are increasingly interested in or required to assess residents using Accreditation Council for Graduate Medical Education (ACGME) milestones and specialty-defined entrustable professional activities (EPAs). The authors aimed to develop a model to predict individual residents' milestone levels based on their assigned EPA entrustment-supervision levels.
Method: During 3 academic years from 2021 to 2024, the authors conducted a multisite prospective cohort study at 48 U.S. pediatric residency programs. Programs collected entrustment-supervision levels for the 17 general pediatrics EPAs and milestone levels for the 22 ACGME pediatric milestones for every resident biannually. EPA and milestone ratings were assigned by clinical competency committees. The first 4 of 6 biannual data reporting cycles were used to fit multilevel structural equation models and produce equations to generate, for each resident, predicted milestone levels based on EPA entrustment-supervision levels. They developed 2 models: one using 17 EPAs and one using 12 EPAs.
Results: Data used for modeling represented 4,328 residents, with 164,886 total entrustment-supervision levels across the general pediatrics EPAs and 243,949 total milestone levels across the pediatric milestones. The fit of the round 1 to 4 model to the round 1 to 4 data (internal prediction) was excellent for both models, with comparative fit indexes of 0.982 (17 EPAs) and 0.981 (12 EPAs). The ability of the round 1 to 4 model to predict milestones for reporting cycles 5 and 6 (external prediction) was similar to the internal predictions, with correlation coefficients of 0.68 (17 EPAs) and 0.69 (12 EPAs) for round 5 and 0.72 (17 EPAs) and 0.68 (12 EPAs) for round 6.
Conclusions: This study demonstrates a strong ability to predict milestone levels based on EPA entrustment-supervision levels in a manner that enables meaningful use of EPAs and milestones in assessment efforts at residency programs.
期刊介绍：
Academic Medicine, the official peer-reviewed journal of the Association of American Medical Colleges, acts as an international forum for exchanging ideas, information, and strategies to address the significant challenges in academic medicine. The journal covers areas such as research, education, clinical care, community collaboration, and leadership, with a commitment to serving the public interest.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
