To the Point: Substituting SOAP Notes for Vignettes in Preclinical Assessment Question Stems.

IF 1.8 Q2 EDUCATION, SCIENTIFIC DISCIPLINES
Medical Science Educator Pub Date : 2025-02-15 eCollection Date: 2025-06-01 DOI:10.1007/s40670-025-02319-6
Kristina Lindquist, Derek Meeks, Kyle Mefferd, Cheryl Vanier, Terrence W Miller
{"title":"To the Point: Substituting SOAP Notes for Vignettes in Preclinical Assessment Question Stems.","authors":"Kristina Lindquist, Derek Meeks, Kyle Mefferd, Cheryl Vanier, Terrence W Miller","doi":"10.1007/s40670-025-02319-6","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>Vignette-based, multiple-choice questions (MCQs) are common in medical school assessments and licensing exams. Ideally each MCQ assesses a specific objective, but question stems may inadvertently assess reading speed and comprehension by introducing construct-irrelevant variance (CIV). The authors hypothesized that question stems written as a Subjective, Objective, Assessment, and Plan (SOAP) note would address this by reducing CIV resulting in a reduction in time students needed to answer the question.</p><p><strong>Methods: </strong>Vignettes were re-written with the same content presented as a SOAP note. Each of ten exams administered to second-year osteopathic medical students over the academic year possessed four questions in which half the students were tested using a vignette and half the SOAP note version. Data were analyzed using a multi-level, mixed-effects model with question type the main effect; question difficulty, student academic rank, and time during the academic year of exam administration served as covariates. Additional analyses included comparisons of mean difficulty indices and discrimination indices for the two question types.</p><p><strong>Results: </strong>Students averaged 3.77 s longer on each vignette-based question compared to the SOAP note counterparts (<i>p</i> < 0.018). Average difficulty indices and point biserials did not differ between SOAP note- and vignette-based questions. Time per question did not change significantly over the year, nor did it vary significantly among students in different academic quartiles.</p><p><strong>Conclusions: </strong>Reduced CIV in the SOAP note stem sharpens the focus of the question on assessing a specific objective, resulting in students spending less time answering the question.</p>","PeriodicalId":37113,"journal":{"name":"Medical Science Educator","volume":"35 3","pages":"1423-1430"},"PeriodicalIF":1.8000,"publicationDate":"2025-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12228897/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Medical Science Educator","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s40670-025-02319-6","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/6/1 0:00:00","PubModel":"eCollection","JCR":"Q2","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}
引用次数: 0

Abstract

Introduction: Vignette-based, multiple-choice questions (MCQs) are common in medical school assessments and licensing exams. Ideally each MCQ assesses a specific objective, but question stems may inadvertently assess reading speed and comprehension by introducing construct-irrelevant variance (CIV). The authors hypothesized that question stems written as a Subjective, Objective, Assessment, and Plan (SOAP) note would address this by reducing CIV resulting in a reduction in time students needed to answer the question.

Methods: Vignettes were re-written with the same content presented as a SOAP note. Each of ten exams administered to second-year osteopathic medical students over the academic year possessed four questions in which half the students were tested using a vignette and half the SOAP note version. Data were analyzed using a multi-level, mixed-effects model with question type the main effect; question difficulty, student academic rank, and time during the academic year of exam administration served as covariates. Additional analyses included comparisons of mean difficulty indices and discrimination indices for the two question types.

Results: Students averaged 3.77 s longer on each vignette-based question compared to the SOAP note counterparts (p < 0.018). Average difficulty indices and point biserials did not differ between SOAP note- and vignette-based questions. Time per question did not change significantly over the year, nor did it vary significantly among students in different academic quartiles.

Conclusions: Reduced CIV in the SOAP note stem sharpens the focus of the question on assessing a specific objective, resulting in students spending less time answering the question.

切中要害:用SOAP笔记代替临床前评估题干中的小插曲。
简介:以小短文为基础的多项选择题(mcq)在医学院评估和执照考试中很常见。理想情况下,每个MCQ评估一个特定的目标,但问题系统可能会不经意地通过引入与结构无关的方差(CIV)来评估阅读速度和理解能力。作者假设,以主观、客观、评估和计划(SOAP)注释形式编写的题干可以通过减少CIV来解决这个问题,从而减少学生回答问题所需的时间。方法:重新编写小插图,并将相同的内容呈现为SOAP注释。在整个学年中,对二年级骨科医学学生进行的10次考试中,每一次都有4个问题,其中一半的学生使用小短文进行测试,另一半使用SOAP笔记版本。采用以问题型为主效应的多层次混合效应模型对数据进行分析;题目难度、学生学术等级和考试管理学年的时间作为协变量。其他分析包括对两种问题类型的平均难度指数和辨别指数的比较。结果:与SOAP笔记相比,学生在每个基于小短文的问题上的平均时间要长3.77秒(p结论:SOAP笔记系统中减少的CIV使问题的重点更加集中在评估特定目标上,导致学生花更少的时间回答问题。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Medical Science Educator
Medical Science Educator Social Sciences-Education
CiteScore
2.90
自引率
11.80%
发文量
202
期刊介绍: Medical Science Educator is the successor of the journal JIAMSE. It is the peer-reviewed publication of the International Association of Medical Science Educators (IAMSE). The Journal offers all who teach in healthcare the most current information to succeed in their task by publishing scholarly activities, opinions, and resources in medical science education. Published articles focus on teaching the sciences fundamental to modern medicine and health, and include basic science education, clinical teaching, and the use of modern education technologies. The Journal provides the readership a better understanding of teaching and learning techniques in order to advance medical science education.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信