Anesthesiology Milestones 1.0: Residents' Learning Trajectories and Training Outcomes.

BACKGROUND The Accreditation Council for Graduate Medical Education (ACGME) requires anesthesiology training programs to submit Milestones ratings for each resident every 6 months, starting in the 2014 to 2015 academic year. We aimed to understand how anesthesiology residents progress through the Milestones evaluations from residency entry to graduation. METHODS Anesthesiology Milestones 1.0 included 25 subcompetencies in 6 core competencies of Patient Care, Medical Knowledge, Systems-Based Practice, Practice-Based Learning & Improvement, Professionalism, and Interpersonal & Communication Skills. Milestones data from all anesthesiology residency programs from July 2015 to June 2019 were used to (1) assess the prevalence of straight-lining (ie, percentage of residents who received the same rating for all 25 subcompetencies for each program per reporting period), (2) describe residents' progression trajectories in each subcompetency, including the baseline at 6-month training, growth rate (ie, rate increase every 12 months), and outcomes on graduation (ie, percentage of residency graduates who received at least 1 Level 4 [target for graduation] or above subcompetency rating, and percentage of graduates who did not reach Level 4 for at least 1 subcompetency), and (3) use 3-level linear mixed-effect models to examine individual resident growth in each subcompetency over time while accounting for clustering effect of residency programs. RESULTS The analyses included 11,691 residents from 153 training programs. Among 6696 clinical anesthesia year 3 residents, 98.3% received at least 1 Level 4 or above rating on graduation; 28.8% to 36.8% of the graduating cohort did not reach Level 4 for at least 1 subcompetency among the 6 core competencies. One hundred and thirty-six programs (88.9% of 153) had straight-lining for at least 1 reporting period. For the multilevel linear mixed-effect models, the fixed intercept estimates were close to 1.0 for Professionalism and Interpersonal & Communication Skills subcompetencies while those for Patient Care subcompetencies were <0.75; the fixed slope estimates were near 1.0 for all 25 subcompetencies. Across all subcompetencies, residency programs accounted for 63% to 84% of between-resident variability in the baseline ratings and 60% to 92% of between-resident variability in the growth rates. CONCLUSIONS We found a high prevalence of straight-lining in Anesthesiology Milestones 1.0 ratings. This raises concerns about whether Milestones evaluations reflect individual resident performance in specific domains. The programs explained most of the between-resident variability in both baseline ratings and growth rates. Future studies are needed to promote standardization and consistency of Milestones evaluations, allowing learners to track their progress and target specific areas to improve performance.