Learner and Teacher Entrustability Concordance in Endocrine Surgery Entrustable Professional Activities.

Abstract

Introduction: The American Association of Endocrine Surgeons (AAES) drafted Entrustable Professional Activities (EPAs) for Endocrine Surgery (ES) to assess general surgery residents and ES fellows in core topics. Here, we examine learner and teacher concordance of entrustability ratings.

Methods: Fourteen EPAs were defined (10 "core"-6 with pre-, intra-, and post-operative phases, four single-phase-and four elective EPAs). Ten institutions collected three-item micro-assessments (MAs) of learner EPA performance using a web-based platform, where entrustment was assessed on observable behaviors using a five-point scale. Learner provided self-assessment ratings, while teachers separately assigned ratings for the same activities. Spearman rank correlation tests were applied to compare learner-teacher ratings for EPAs.

Results: MAs (n = 698) were submitted between 7/2022-9/2023. Programs submitted between 3 and 468 MAs. Among 3-phase core EPAs, the intraoperative phase was most assessed (n = 496), followed by pre- and post-operative phases (n = 108 and 55, respectively). Thirty-nine single-phase EPAs were submitted. Learner self-assessment and teacher ratings were equivalent for 32% of pre-operative EPAs, 53% of intra-operative EPAs, 29% of post-operative EPAs, and 64% of single phase of care EPAs. Spearman rank correlation between teacher and learner ratings for all phases of care combined was 0.66. Correlation was highest for the intra-operative phase (0.73), followed by single-phase (0.65), pre-operative phase (0.27), and post-operative phase (0.20). Rating concordance did not differ when high and low scores were examined independently. Learners were more prone to underestimate performance than to overestimate it when ratings were discordant.

Conclusions: Learners' self-assessment and teacher ratings of endocrine surgery EPAs are strongly concordant with operative technical skills. However, learners often underestimate their performance of pre- and post-operative care compared to faculty ratings when using EPAs.