{"title":"Framing our Expectations: Variability in Entrustable Professional Activity Assessments","authors":"","doi":"10.1016/j.jsurg.2024.07.025","DOIUrl":null,"url":null,"abstract":"<div><h3>OBJECTIVE</h3><p>To determine the ability of surgical trainees and faculty to correctly interpret entrustability of a resident learner in a modeled patient care scenario.</p></div><div><h3>DESIGN</h3><p>Prospective study utilizing a web-based survey including 4 previously-recorded short videos of resident learners targeted to specific levels of the American Board of Surgery's (ABS) Entrustment Scale. Respondents were asked to choose the entrustment level that best corresponded to their observations of the learner in the video. Responses were subcategorized by low and high entrustment.</p></div><div><h3>SETTING</h3><p>Online, utilizing the Qualtrics survey platform.</p></div><div><h3>PARTICIPANTS</h3><p>Survey targeting US surgical trainees and surgical faculty via email and social media. We received 31 complete responses and 2 responses which completed > 1 video assessment question without demographic information (n = 33). Respondents included 10 trainees (32%) and 21 attending surgeons (68%).</p></div><div><h3>RESULTS</h3><p>Neither faculty nor trainees readily identified the targeted entrustment level for Question 1 (preoperative care of a patient with acute appendicitis with high entrustment, 36% correct), though evaluations of the remaining questions (2 through 4) demonstrated more accuracy (70, 84, and 75% correct, respectively). Faculty were more readily able than trainees to identify low entrustment (level Limited Participation) in intraoperative inguinal hernia repair (95% vs 60%, p = 0.03). After subcategorization to high and low entrustment, both residents and faculty were able to accurately identify entrustment 95% overall.</p></div><div><h3>CONCLUSIONS</h3><p>Both trainees and attending surgeons were able to identify high- and low-performing residents on short video demonstrations using the ABS EPA entrustment scale. This provides additional evidence in support of the need for frequent observations of EPAs to account for the variability in raters’ perceptions in addition to complexity of clinical scenarios. Frame-of-reference training via a video-based platform may also be beneficial for both residents and faculty as an ongoing EPA implementation strategy.</p></div>","PeriodicalId":50033,"journal":{"name":"Journal of Surgical Education","volume":null,"pages":null},"PeriodicalIF":2.6000,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Surgical Education","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1931720424003490","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}
引用次数: 0
Abstract
OBJECTIVE
To determine the ability of surgical trainees and faculty to correctly interpret entrustability of a resident learner in a modeled patient care scenario.
DESIGN
Prospective study utilizing a web-based survey including 4 previously-recorded short videos of resident learners targeted to specific levels of the American Board of Surgery's (ABS) Entrustment Scale. Respondents were asked to choose the entrustment level that best corresponded to their observations of the learner in the video. Responses were subcategorized by low and high entrustment.
SETTING
Online, utilizing the Qualtrics survey platform.
PARTICIPANTS
Survey targeting US surgical trainees and surgical faculty via email and social media. We received 31 complete responses and 2 responses which completed > 1 video assessment question without demographic information (n = 33). Respondents included 10 trainees (32%) and 21 attending surgeons (68%).
RESULTS
Neither faculty nor trainees readily identified the targeted entrustment level for Question 1 (preoperative care of a patient with acute appendicitis with high entrustment, 36% correct), though evaluations of the remaining questions (2 through 4) demonstrated more accuracy (70, 84, and 75% correct, respectively). Faculty were more readily able than trainees to identify low entrustment (level Limited Participation) in intraoperative inguinal hernia repair (95% vs 60%, p = 0.03). After subcategorization to high and low entrustment, both residents and faculty were able to accurately identify entrustment 95% overall.
CONCLUSIONS
Both trainees and attending surgeons were able to identify high- and low-performing residents on short video demonstrations using the ABS EPA entrustment scale. This provides additional evidence in support of the need for frequent observations of EPAs to account for the variability in raters’ perceptions in addition to complexity of clinical scenarios. Frame-of-reference training via a video-based platform may also be beneficial for both residents and faculty as an ongoing EPA implementation strategy.
期刊介绍:
The Journal of Surgical Education (JSE) is dedicated to advancing the field of surgical education through original research. The journal publishes research articles in all surgical disciplines on topics relative to the education of surgical students, residents, and fellows, as well as practicing surgeons. Our readers look to JSE for timely, innovative research findings from the international surgical education community. As the official journal of the Association of Program Directors in Surgery (APDS), JSE publishes the proceedings of the annual APDS meeting held during Surgery Education Week.