Reliability, validity and acceptability of an online clinical reasoning simulator for medical students: An international pilot.

IF 3.3 2区教育学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

Medical Teacher Pub Date : 2024-09-01 Epub Date: 2024-03-15 DOI:10.1080/0142159X.2024.2308082

Eduardo Hornos, Eduardo Pleguezuelos, Laksha Bala, Carlos Fernando Collares, Adrian Freeman, Cees van der Vleuten, Kevin G Murphy, Amir H Sam

{"title":"Reliability, validity and acceptability of an online clinical reasoning simulator for medical students: An international pilot.","authors":"Eduardo Hornos, Eduardo Pleguezuelos, Laksha Bala, Carlos Fernando Collares, Adrian Freeman, Cees van der Vleuten, Kevin G Murphy, Amir H Sam","doi":"10.1080/0142159X.2024.2308082","DOIUrl":null,"url":null,"abstract":"Introduction: Clinical reasoning skills are essential for decision-making. Current assessment methods are limited when testing clinical reasoning and management of uncertainty. This study evaluates the reliability, validity and acceptability of Practicum Script, an online simulation-based programme, for developing medical students' clinical reasoning skills using real-life cases.Methods: In 2020, we conducted an international, multicentre pilot study using 20 clinical cases with 2457 final-year medical students from 21 schools worldwide. Psychometric analysis was performed (n = 1502 students completing at least 80% of cases). Classical estimates of reliability for three test domains (hypothesis generation, hypothesis argumentation and knowledge application) were calculated using Cronbach's alpha and McDonald's omega coefficients. Validity evidence was obtained by confirmatory factor analysis (CFA) and measurement alignment (MA). Items from the knowledge application domain were analysed using cognitive diagnostic modelling (CDM). Acceptability was evaluated by an anonymous student survey.Results: Reliability estimates were high with narrow confidence intervals. CFA revealed acceptable goodness-of-fit indices for the proposed three-factor model. CDM analysis demonstrated good absolute test fit and high classification accuracy estimates. Student survey responses showed high levels of acceptability.Conclusion: Our findings suggest that Practicum Script is a useful resource for strengthening students' clinical reasoning skills and ability to manage uncertainty.","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":null,"pages":null},"PeriodicalIF":3.3000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Medical Teacher","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/0142159X.2024.2308082","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/3/15 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction: Clinical reasoning skills are essential for decision-making. Current assessment methods are limited when testing clinical reasoning and management of uncertainty. This study evaluates the reliability, validity and acceptability of Practicum Script, an online simulation-based programme, for developing medical students' clinical reasoning skills using real-life cases.

Methods: In 2020, we conducted an international, multicentre pilot study using 20 clinical cases with 2457 final-year medical students from 21 schools worldwide. Psychometric analysis was performed (n = 1502 students completing at least 80% of cases). Classical estimates of reliability for three test domains (hypothesis generation, hypothesis argumentation and knowledge application) were calculated using Cronbach's alpha and McDonald's omega coefficients. Validity evidence was obtained by confirmatory factor analysis (CFA) and measurement alignment (MA). Items from the knowledge application domain were analysed using cognitive diagnostic modelling (CDM). Acceptability was evaluated by an anonymous student survey.

Results: Reliability estimates were high with narrow confidence intervals. CFA revealed acceptable goodness-of-fit indices for the proposed three-factor model. CDM analysis demonstrated good absolute test fit and high classification accuracy estimates. Student survey responses showed high levels of acceptability.

Conclusion: Our findings suggest that Practicum Script is a useful resource for strengthening students' clinical reasoning skills and ability to manage uncertainty.

查看原文本刊更多论文

医学生在线临床推理模拟器的可靠性、有效性和可接受性：国际试点。

引言临床推理技能对决策至关重要。目前的评估方法在测试临床推理和不确定性管理方面存在局限性。本研究评估了 Practicum Script 这一在线模拟程序的可靠性、有效性和可接受性，该程序利用真实病例培养医学生的临床推理能力：2020 年，我们对来自全球 21 所学校的 2457 名应届医学生进行了一项国际多中心试点研究，使用了 20 个临床病例。我们进行了心理测量分析（n = 1502 名学生完成了至少 80% 的病例）。使用 Cronbach's alpha 和 McDonald's omega 系数计算了三个测试领域（假设产生、假设论证和知识应用）的经典可靠性估计值。通过确认性因素分析（CFA）和测量排列（MA）获得了有效性证据。采用认知诊断模型（CDM）对知识应用领域的项目进行了分析。通过匿名学生调查对可接受性进行了评估：结果：信度估计值较高，置信区间较窄。CFA显示，所提出的三因素模型的拟合优度指数可以接受。CDM分析表明绝对测试拟合度高，分类准确度估计值高。学生的调查反馈显示了较高的可接受性：我们的研究结果表明，实践脚本是加强学生临床推理技能和管理不确定性能力的有用资源。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Medical Teacher 医学-卫生保健

CiteScore

7.80

自引率

8.50%

发文量

396

审稿时长

3-6 weeks

期刊介绍： Medical Teacher provides accounts of new teaching methods, guidance on structuring courses and assessing achievement, and serves as a forum for communication between medical teachers and those involved in general education. In particular, the journal recognizes the problems teachers have in keeping up-to-date with the developments in educational methods that lead to more effective teaching and learning at a time when the content of the curriculum—from medical procedures to policy changes in health care provision—is also changing. The journal features reports of innovation and research in medical education, case studies, survey articles, practical guidelines, reviews of current literature and book reviews. All articles are peer reviewed.