Can routine EPA-based assessments predict OSCE performances of undergraduate medical students?

IF 3.3 2区教育学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

Medical Teacher Pub Date : 2025-07-01 Epub Date: 2024-10-14 DOI:10.1080/0142159X.2024.2413024

Yi-Hsuan Lin, Ying-Ying Yang, Chen-Huan Chen, Chia-Chang Huang, Chung-Pin Li, Hsiao-Chin Shen, Hsiao-Yun Yeh, Jen-Feng Liang, Shao-Yin Chu, Jiing-Feng Lirng, Shuu-Jiun Wang

{"title":"Can routine EPA-based assessments predict OSCE performances of undergraduate medical students?","authors":"Yi-Hsuan Lin, Ying-Ying Yang, Chen-Huan Chen, Chia-Chang Huang, Chung-Pin Li, Hsiao-Chin Shen, Hsiao-Yun Yeh, Jen-Feng Liang, Shao-Yin Chu, Jiing-Feng Lirng, Shuu-Jiun Wang","doi":"10.1080/0142159X.2024.2413024","DOIUrl":null,"url":null,"abstract":"Background: Objective structured clinical examination (OSCE) is used worldwide. This study aims to explore potential alternatives to the OSCE by using entrustable professional activities (EPA)-based assessments in the workplace.Methods: This study enrolled 265 six-year undergraduate medical students (UGY) from 2021 to 2023. During their rotations, students were assessed using 13 EPAs, with the grading methods modified to facilitate application. Before graduation, they participated in two mock OSCEs and a National OSCE. We used generalized estimating equations to analyze the associations between the EPA assessments and the OSCE scores, adjusting for age and sex, and developed a prediction model. EPA8 and EPA9, which represent advanced abilities that were not significant in the regression models, were removed from the prediction model.Results: Most EPAs were significantly correlated with OSCE scores across the three cohorts. The prediction model for forecasting passing in the three OSCEs demonstrated fair predictive capacity (area under curve = 0.82, 0.66, and 0.71 for students graduated in 2021, 2022, and 2023, respectively all p < 0.05).Conclusions: The workplace-based assessments (EPA) showed a high correlation with competency-based assessments in simulated settings (OSCE). EPAs may serve as alternative tools to formal OSCE for medical students.","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"1134-1143"},"PeriodicalIF":3.3000,"publicationDate":"2025-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Medical Teacher","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/0142159X.2024.2413024","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/10/14 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

Abstract

Background: Objective structured clinical examination (OSCE) is used worldwide. This study aims to explore potential alternatives to the OSCE by using entrustable professional activities (EPA)-based assessments in the workplace.

Methods: This study enrolled 265 six-year undergraduate medical students (UGY) from 2021 to 2023. During their rotations, students were assessed using 13 EPAs, with the grading methods modified to facilitate application. Before graduation, they participated in two mock OSCEs and a National OSCE. We used generalized estimating equations to analyze the associations between the EPA assessments and the OSCE scores, adjusting for age and sex, and developed a prediction model. EPA8 and EPA9, which represent advanced abilities that were not significant in the regression models, were removed from the prediction model.

Results: Most EPAs were significantly correlated with OSCE scores across the three cohorts. The prediction model for forecasting passing in the three OSCEs demonstrated fair predictive capacity (area under curve = 0.82, 0.66, and 0.71 for students graduated in 2021, 2022, and 2023, respectively all p < 0.05).

Conclusions: The workplace-based assessments (EPA) showed a high correlation with competency-based assessments in simulated settings (OSCE). EPAs may serve as alternative tools to formal OSCE for medical students.

查看原文本刊更多论文

基于 EPA 的常规评估能否预测医科本科生的 OSCE 成绩？

背景：客观结构化临床考试（OSCE）已在全球范围内使用。本研究旨在通过在工作场所使用基于可委托专业活动（EPA）的评估，探索 OSCE 的潜在替代方法：本研究从 2021 年至 2023 年招收了 265 名六年制医学本科生（UGY）。在轮转期间，学生们接受了 13 项 EPA 的评估，为便于应用，对评分方法进行了修改。毕业前，他们参加了两次模拟 OSCE 和一次全国 OSCE。我们使用广义估计方程分析了 EPA 评估与 OSCE 分数之间的关联，并对年龄和性别进行了调整，建立了一个预测模型。代表高级能力的 EPA8 和 EPA9 在回归模型中不显著，因此从预测模型中删除：结果：在三批学生中，大多数 EPA 与 OSCE 分数有明显的相关性。预测三项 OSCE 考试合格的预测模型显示出相当的预测能力（2021、2022 和 2023 年毕业的学生的曲线下面积分别为 0.82、0.66 和 0.71，均为 p 结论：基于工作场所的评估（EPA）与模拟环境中基于能力的评估（OSCE）具有很高的相关性。EPA 可作为医学生正式 OSCE 的替代工具。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Medical Teacher 医学-卫生保健

CiteScore

7.80

自引率

8.50%

发文量

396

审稿时长

3-6 weeks

期刊介绍： Medical Teacher provides accounts of new teaching methods, guidance on structuring courses and assessing achievement, and serves as a forum for communication between medical teachers and those involved in general education. In particular, the journal recognizes the problems teachers have in keeping up-to-date with the developments in educational methods that lead to more effective teaching and learning at a time when the content of the curriculum—from medical procedures to policy changes in health care provision—is also changing. The journal features reports of innovation and research in medical education, case studies, survey articles, practical guidelines, reviews of current literature and book reviews. All articles are peer reviewed.