Evaluation of assessment marks in the clinical years of an undergraduate medical training programme: Where are we and how can we improve?

IF 0.4 Q4 HEALTH CARE SCIENCES & SERVICES
H. Brits, G. Joubert, J. Bezuidenhout, L. J. van der Merwe
{"title":"Evaluation of assessment marks in the clinical years of an undergraduate medical training programme: Where are we and how can we improve?","authors":"H. Brits, G. Joubert, J. Bezuidenhout, L. J. van der Merwe","doi":"10.7196/ajhpe.2021.v13i4.1379","DOIUrl":null,"url":null,"abstract":"Background. In high-stakes assessments, the accuracy and consistency of the decision to pass or fail a student is as important as the reliability of the assessment. Objective. To evaluate the reliability of results of high-stakes assessments in the clinical phase of the undergraduate medical programme at the University of the Free State, as a step to make recommendations for improving quality assessment. Methods. A cohort analytical study design was used. The final, end-of-block marks and the end-of-year assessment marks of both fourth-year and final-year medical students over 3 years were compared for decision reliability, test-retest reliability, stability and reproducibility. Results. 1 380 marks in 26 assessments were evaluated. The G-index of agreement for decision reliability ranged from 0.86 to 0.98. In 88.9% of assessments, the test-retest correlation coefficient was <0.7. Mean marks for end-of-block and end-of-year assessments were similar. However, the standard deviations of differences between end-of-block and end-of-year assessment marks were high. Multiple-choice questions (MCQs) and objective structured clinical examinations (OSCEs) yielded good reliability results. Conclusion. The reliability of pass/fail outcome decisions was good. The test reliability, as well as stability and reproducibility of individual student marks, could not be accurately replicated. The use of MCQs and OSCEs are practical examples of where the number of assessments can be increased to improve reliability. In order to increase the number of assessments and to reduce the stress of high-stake assessments, more workplace-based assessment with observed clinical cases is recommended.","PeriodicalId":43683,"journal":{"name":"African Journal of Health Professions Education","volume":" ","pages":""},"PeriodicalIF":0.4000,"publicationDate":"2021-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"African Journal of Health Professions Education","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.7196/ajhpe.2021.v13i4.1379","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

Abstract

Background. In high-stakes assessments, the accuracy and consistency of the decision to pass or fail a student is as important as the reliability of the assessment. Objective. To evaluate the reliability of results of high-stakes assessments in the clinical phase of the undergraduate medical programme at the University of the Free State, as a step to make recommendations for improving quality assessment. Methods. A cohort analytical study design was used. The final, end-of-block marks and the end-of-year assessment marks of both fourth-year and final-year medical students over 3 years were compared for decision reliability, test-retest reliability, stability and reproducibility. Results. 1 380 marks in 26 assessments were evaluated. The G-index of agreement for decision reliability ranged from 0.86 to 0.98. In 88.9% of assessments, the test-retest correlation coefficient was <0.7. Mean marks for end-of-block and end-of-year assessments were similar. However, the standard deviations of differences between end-of-block and end-of-year assessment marks were high. Multiple-choice questions (MCQs) and objective structured clinical examinations (OSCEs) yielded good reliability results. Conclusion. The reliability of pass/fail outcome decisions was good. The test reliability, as well as stability and reproducibility of individual student marks, could not be accurately replicated. The use of MCQs and OSCEs are practical examples of where the number of assessments can be increased to improve reliability. In order to increase the number of assessments and to reduce the stress of high-stake assessments, more workplace-based assessment with observed clinical cases is recommended.
本科医学培训课程临床学年考核分数的评估:我们在哪里?我们如何改进?
背景。在高风险的评估中,决定学生及格或不及格的准确性和一致性与评估的可靠性同样重要。目标。评估自由邦大学本科医学方案临床阶段高风险评估结果的可靠性,作为提出改进质量评估建议的一个步骤。方法。采用队列分析研究设计。比较四年级医学生和大四医学生三年内的期末、期末块分和期末评估分的决策信度、重测信度、稳定性和再现性。结果:共评定26个单项的1380分。决策信度一致性g指数为0.86 ~ 0.98。在88.9%的评估中,重测相关系数<0.7。期末和年度评估的平均分数相似。然而,期末和年度评估成绩差异的标准差较高。多项选择题(mcq)和客观结构化临床检查(oses)获得了良好的可靠性结果。结论。通过/失败结果决策的可靠性很好。测试的信度,以及个别学生分数的稳定性和可重复性,都不能被准确地复制。mcq和osce的使用是可以增加评估数量以提高可靠性的实际示例。为了增加评估的数量并减少高风险评估的压力,建议更多地以工作场所为基础,结合观察到的临床病例进行评估。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
African Journal of Health Professions Education
African Journal of Health Professions Education HEALTH CARE SCIENCES & SERVICES-
自引率
0.00%
发文量
18
审稿时长
24 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信