Home Field Advantage? Comparing the Quality of EPA Observations Completed On- vs Off-Service.

Journal of graduate medical education Pub Date : 2025-08-01 Epub Date: 2025-08-15 DOI:10.4300/JGME-D-24-00719.1
Kaitlin Endres, Timothy J Wood, Nancy Dudek, Warren J Cheung
{"title":"Home Field Advantage? Comparing the Quality of EPA Observations Completed On- vs Off-Service.","authors":"Kaitlin Endres, Timothy J Wood, Nancy Dudek, Warren J Cheung","doi":"10.4300/JGME-D-24-00719.1","DOIUrl":null,"url":null,"abstract":"<p><p><b>Background</b> Increasing evidence suggests that supervisors exhibit different assessment behaviors for residents from their own discipline compared to those completing a rotation from another discipline. As programs of assessment rely on collecting robust performance data to inform high-stakes decisions about progress and promotion, it is important to examine the quality of such inputs. <b>Objective</b> To compare the quality of workplace-based assessments (WBAs) of emergency medicine (EM) residents by EM and non-EM assessors. <b>Methods</b> This retrospective database study compared the quality of WBAs using the Quality of Assessment of Learning (QuAL) score (range 0-5), a previously published measure of WBA quality that has demonstrated strong psychometric characteristics. Five entrustable professional activities (EPAs), 3 procedural and 2 non-procedural, mapped to both EM and non-EM rotations, were selected for inclusion. Two hundred and fifty WBAs (50 WBAs per EPA; 25 EM and 25 non-EM), completed from July 2019 to June 2021, were rated by 3 blinded EM physician raters. QuAL scores were analysed using factorial ANOVA. <b>Results</b> Mean QuAL scores for WBAs completed during EM rotations were significantly higher compared to those completed during non-EM rotations (3.66±0.99 vs 3.02±0.99). Further, mean QuAL score for procedural EPAs was significantly higher than non-procedural EPAs (3.61±1.00 vs 3.16±1.03). <b>Conclusions</b> In this study, the quality of WBAs completed for EM residents during non-EM rotations was of lower quality compared to assessments on EM rotations.</p>","PeriodicalId":37886,"journal":{"name":"Journal of graduate medical education","volume":"17 4","pages":"464-469"},"PeriodicalIF":0.0000,"publicationDate":"2025-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12360236/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of graduate medical education","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.4300/JGME-D-24-00719.1","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/8/15 0:00:00","PubModel":"Epub","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Background Increasing evidence suggests that supervisors exhibit different assessment behaviors for residents from their own discipline compared to those completing a rotation from another discipline. As programs of assessment rely on collecting robust performance data to inform high-stakes decisions about progress and promotion, it is important to examine the quality of such inputs. Objective To compare the quality of workplace-based assessments (WBAs) of emergency medicine (EM) residents by EM and non-EM assessors. Methods This retrospective database study compared the quality of WBAs using the Quality of Assessment of Learning (QuAL) score (range 0-5), a previously published measure of WBA quality that has demonstrated strong psychometric characteristics. Five entrustable professional activities (EPAs), 3 procedural and 2 non-procedural, mapped to both EM and non-EM rotations, were selected for inclusion. Two hundred and fifty WBAs (50 WBAs per EPA; 25 EM and 25 non-EM), completed from July 2019 to June 2021, were rated by 3 blinded EM physician raters. QuAL scores were analysed using factorial ANOVA. Results Mean QuAL scores for WBAs completed during EM rotations were significantly higher compared to those completed during non-EM rotations (3.66±0.99 vs 3.02±0.99). Further, mean QuAL score for procedural EPAs was significantly higher than non-procedural EPAs (3.61±1.00 vs 3.16±1.03). Conclusions In this study, the quality of WBAs completed for EM residents during non-EM rotations was of lower quality compared to assessments on EM rotations.

主场优势?比较在服务和关闭服务时完成的EPA观测的质量。
背景越来越多的证据表明,与从其他学科完成轮转的住院医生相比,督导对来自本学科的住院医生表现出不同的评估行为。由于评估项目依赖于收集可靠的绩效数据,为有关进展和晋升的高风险决策提供信息,因此检查此类输入的质量非常重要。目的比较急诊医师(EM)与非急诊医师(EM)工作场所评价(WBAs)的质量。方法:本回顾性数据库研究使用学习质量评估(quality of Assessment of Learning, QuAL)评分(范围0-5)比较了WBA的质量,这是一种先前发表的WBA质量测量方法,显示出强烈的心理测量特征。五个可委托的专业活动(EPAs), 3个程序性和2个非程序性,映射到EM和非EM轮转,被选中纳入。从2019年7月到2021年6月,由3名盲法EM医生评分者对250份wba(每个EPA 50份wba; 25份EM和25份非EM)进行评分。使用因子方差分析对质量评分进行分析。结果在EM旋转期间完成的wba的平均质量分数明显高于非EM旋转期间完成的wba(3.66±0.99 vs 3.02±0.99)。此外,程序性EPAs的平均QuAL评分显著高于非程序性EPAs(3.61±1.00 vs 3.16±1.03)。在本研究中,与EM轮转评估相比,EM轮转期间EM居民完成的wba质量较低。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of graduate medical education
Journal of graduate medical education Medicine-Medicine (all)
CiteScore
3.20
自引率
0.00%
发文量
248
期刊介绍: - Be the leading peer-reviewed journal in graduate medical education; - Promote scholarship and enhance the quality of research in the field; - Disseminate evidence-based approaches for teaching, assessment, and improving the learning environment; and - Generate new knowledge that enhances graduates'' ability to provide high-quality, cost-effective care.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信