Evaluating a Global Assessment Measure Created by Standardized Patients for the Multiple Mini Interview in Medical School Admissions: Mixed Methods Study.

Q2 Medicine
Ann Blair Kennedy, Cindy Nessim Youssef Riyad, Ryan Ellis, Perry R Fleming, Mallorie Gainey, Kara Templeton, Anna Nourse, Virginia Hardaway, April Brown, Pam Evans, Nabil Natafgi
{"title":"Evaluating a Global Assessment Measure Created by Standardized Patients for the Multiple Mini Interview in Medical School Admissions: Mixed Methods Study.","authors":"Ann Blair Kennedy, Cindy Nessim Youssef Riyad, Ryan Ellis, Perry R Fleming, Mallorie Gainey, Kara Templeton, Anna Nourse, Virginia Hardaway, April Brown, Pam Evans, Nabil Natafgi","doi":"10.2196/38209","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Standardized patients (SPs) are essential stakeholders in the multiple mini interviews (MMIs) that are increasingly used to assess medical school applicants' interpersonal skills. However, there is little evidence for their inclusion in the development of instruments.</p><p><strong>Objective: </strong>This study aimed to describe the process and evaluate the impact of having SPs co-design and cocreate a global measurement question that assesses medical school applicants' readiness for medical school and acceptance status.</p><p><strong>Methods: </strong>This study used an exploratory, sequential, and mixed methods study design. First, we evaluated the initial MMI program and determined the next quality improvement steps. Second, we held a collaborative workshop with SPs to codevelop the assessment question and response options. Third, we evaluated the created question and the additional MMI rubric items through statistical tests based on 1084 applicants' data from 3 cohorts of applicants starting in the 2018-2019 academic year. The internal reliability of the MMI was measured using a Cronbach α test, and its prediction of admission status was tested using a forward stepwise binary logistic regression.</p><p><strong>Results: </strong>Program evaluation indicated the need for an additional quantitative question to assess applicant readiness for medical school. In total, 3 simulation specialists, 2 researchers, and 21 SPs participated in a workshop leading to a final global assessment question and responses. The Cronbach α's were >0.8 overall and in each cohort year. The final stepwise logistic model for all cohorts combined was statistically significant (P<.001), explained 9.2% (R<sup>2</sup>) of the variance in acceptance status, and correctly classified 65.5% (637/972) of cases. The final model consisted of 3 variables: empathy, rank of readiness, and opening the encounter.</p><p><strong>Conclusions: </strong>The collaborative nature of this project between stakeholders, including nonacademics and researchers, was vital for the success of this project. The SP-created question had a significant impact on the final model predicting acceptance to medical school. This finding indicates that SPs bring a critical perspective that can improve the process of evaluating medical school applicants.</p>","PeriodicalId":36208,"journal":{"name":"Journal of Participatory Medicine","volume":"14 1","pages":"e38209"},"PeriodicalIF":0.0000,"publicationDate":"2022-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9472042/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Participatory Medicine","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.2196/38209","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Medicine","Score":null,"Total":0}
引用次数: 0

Abstract

Background: Standardized patients (SPs) are essential stakeholders in the multiple mini interviews (MMIs) that are increasingly used to assess medical school applicants' interpersonal skills. However, there is little evidence for their inclusion in the development of instruments.

Objective: This study aimed to describe the process and evaluate the impact of having SPs co-design and cocreate a global measurement question that assesses medical school applicants' readiness for medical school and acceptance status.

Methods: This study used an exploratory, sequential, and mixed methods study design. First, we evaluated the initial MMI program and determined the next quality improvement steps. Second, we held a collaborative workshop with SPs to codevelop the assessment question and response options. Third, we evaluated the created question and the additional MMI rubric items through statistical tests based on 1084 applicants' data from 3 cohorts of applicants starting in the 2018-2019 academic year. The internal reliability of the MMI was measured using a Cronbach α test, and its prediction of admission status was tested using a forward stepwise binary logistic regression.

Results: Program evaluation indicated the need for an additional quantitative question to assess applicant readiness for medical school. In total, 3 simulation specialists, 2 researchers, and 21 SPs participated in a workshop leading to a final global assessment question and responses. The Cronbach α's were >0.8 overall and in each cohort year. The final stepwise logistic model for all cohorts combined was statistically significant (P<.001), explained 9.2% (R2) of the variance in acceptance status, and correctly classified 65.5% (637/972) of cases. The final model consisted of 3 variables: empathy, rank of readiness, and opening the encounter.

Conclusions: The collaborative nature of this project between stakeholders, including nonacademics and researchers, was vital for the success of this project. The SP-created question had a significant impact on the final model predicting acceptance to medical school. This finding indicates that SPs bring a critical perspective that can improve the process of evaluating medical school applicants.

Abstract Image

Abstract Image

评估由标准化患者为医学院入学考试中的多重小型面试创建的全球评估量表:混合方法研究。
背景:标准化病人(SPs)是多重小型面试(MMIs)中不可或缺的利益相关者,这种面试越来越多地被用于评估医学院申请者的人际交往能力。然而,很少有证据表明将他们纳入了工具的开发中:本研究旨在描述让特殊学生共同设计和共同创造一个全球性测量问题的过程并评估其影响,该问题用于评估医学院申请者对医学院的准备情况和录取情况:本研究采用了探索性、连续性和混合方法的研究设计。首先,我们对最初的 MMI 计划进行了评估,并确定了下一步的质量改进措施。其次,我们与 SPs 举行了一次合作研讨会,以编制评估问题和回答选项。第三,我们根据2018-2019学年开始的3批1084名申请人的数据,通过统计测试评估了创建的问题和额外的MMI评分标准项目。我们使用 Cronbach α 检验测量了 MMI 的内部信度,并使用前向逐步二元逻辑回归检验了其对录取状态的预测:结果:项目评估表明,有必要增加一个定量问题来评估申请者是否做好了就读医学院的准备。共有 3 名模拟专家、2 名研究人员和 21 名专科医生参加了研讨会,最终确定了总体评估问题和答案。总体和各年级的 Cronbach α 均大于 0.8。所有组群合并的最终逐步逻辑模型对接受状况的差异具有统计学意义(P2),并对 65.5% 的病例(637/972)进行了正确分类。最终模型由 3 个变量组成:移情、准备程度等级和开始接触:包括非学术界人士和研究人员在内的利益相关者之间的合作对本项目的成功至关重要。由专业医师提出的问题对预测医学院录取率的最终模型有重大影响。这一研究结果表明,SP 带来了一种批判性的视角,可以改善医学院申请者的评估过程。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Participatory Medicine
Journal of Participatory Medicine Medicine-Medicine (miscellaneous)
CiteScore
3.20
自引率
0.00%
发文量
8
审稿时长
12 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信