Assessing medical knowledge: A 3-year comparative study of very short answer vs. multiple choice questions.

IF 3.3 2区教育学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

Medical Teacher Pub Date : 2025-04-28 DOI:10.1080/0142159X.2025.2496382

Harry G Potter, John C McLachlan

{"title":"Assessing medical knowledge: A 3-year comparative study of very short answer vs. multiple choice questions.","authors":"Harry G Potter, John C McLachlan","doi":"10.1080/0142159X.2025.2496382","DOIUrl":null,"url":null,"abstract":"Purpose: Assessment design significantly influences evaluation of student learning. Multiple choice questions (MCQ) and very short answer questions (VSAQ) are commonly used assessment formats, especially in high-stakes settings like medical education. MCQs are favoured for efficiency, coverage, and reliability but may lack depth in assessing critical thinking. VSAQs require students to generate responses, potentially enhancing depth, but posing challenges in consistency and subjective interpretation.Methods: Data from parallel MCQ/VSAQ exams over three years was collected. Summary statistics for each exam (marks, time, and discrimination index; DI) and the effect of year and question characteristics were analysed.Results: VSAQs were associated with lower marks (p < 0.001), longer time (p < 0.001), and higher DI (p < 0.001). Question characteristics (e.g. basic science or clinical stems) significantly affected the mark, time, and DI, changing across years, but not interacting with question format.Conclusion: While MCQs resulted in higher marks, VSAQs provided higher discrimination of student performance. Response options in MCQs likely enhance recall, however real-world settings also offer contextual cues. Question characteristics affect student performance independently of format, likely due to differences in cohort career progression. Future research should investigate predictive validity and standard setting of VSAQs in a basic science context.","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"1-9"},"PeriodicalIF":3.3000,"publicationDate":"2025-04-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Medical Teacher","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/0142159X.2025.2496382","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: Assessment design significantly influences evaluation of student learning. Multiple choice questions (MCQ) and very short answer questions (VSAQ) are commonly used assessment formats, especially in high-stakes settings like medical education. MCQs are favoured for efficiency, coverage, and reliability but may lack depth in assessing critical thinking. VSAQs require students to generate responses, potentially enhancing depth, but posing challenges in consistency and subjective interpretation.

Methods: Data from parallel MCQ/VSAQ exams over three years was collected. Summary statistics for each exam (marks, time, and discrimination index; DI) and the effect of year and question characteristics were analysed.

Results: VSAQs were associated with lower marks (p < 0.001), longer time (p < 0.001), and higher DI (p < 0.001). Question characteristics (e.g. basic science or clinical stems) significantly affected the mark, time, and DI, changing across years, but not interacting with question format.

Conclusion: While MCQs resulted in higher marks, VSAQs provided higher discrimination of student performance. Response options in MCQs likely enhance recall, however real-world settings also offer contextual cues. Question characteristics affect student performance independently of format, likely due to differences in cohort career progression. Future research should investigate predictive validity and standard setting of VSAQs in a basic science context.

查看原文本刊更多论文

评估医学知识：一项为期3年的简答题与多项选择题的比较研究。

目的：评价设计对学生学习评价有显著影响。多项选择题（MCQ）和极简答题（VSAQ）是常用的评估形式，特别是在高风险的环境中，如医学教育。mcq在效率、覆盖面和可靠性方面受到青睐，但在评估批判性思维方面可能缺乏深度。VSAQs要求学生做出回应，这可能会增加深度，但会在一致性和主观解释方面带来挑战。方法：收集3年来MCQ/VSAQ平行考试资料。每次考试的汇总统计（分数、时间、判别指标）；分析了年份和问题特征的影响。结果：VSAQs与较低的分数相关（p p p）结论：MCQs导致较高的分数，而VSAQs对学生的表现有较高的歧视。mcq中的回应选项可能会提高记忆，但现实世界的设置也会提供情境线索。问题特征独立于形式影响学生的表现，可能是由于队列职业发展的差异。未来的研究应在基础科学背景下探讨VSAQs的预测有效性和标准制定。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Medical Teacher 医学-卫生保健

CiteScore

7.80

自引率

8.50%

发文量

396

审稿时长

3-6 weeks

期刊介绍： Medical Teacher provides accounts of new teaching methods, guidance on structuring courses and assessing achievement, and serves as a forum for communication between medical teachers and those involved in general education. In particular, the journal recognizes the problems teachers have in keeping up-to-date with the developments in educational methods that lead to more effective teaching and learning at a time when the content of the curriculum—from medical procedures to policy changes in health care provision—is also changing. The journal features reports of innovation and research in medical education, case studies, survey articles, practical guidelines, reviews of current literature and book reviews. All articles are peer reviewed.