Validation of the Learner Engagement Instrument for Continuing Professional Development.

IF 5.3 2区 教育学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES
Academic Medicine Pub Date : 2024-09-01 Epub Date: 2024-07-29 DOI:10.1097/ACM.0000000000005749
David A Cook, Christopher R Stephenson
{"title":"Validation of the Learner Engagement Instrument for Continuing Professional Development.","authors":"David A Cook, Christopher R Stephenson","doi":"10.1097/ACM.0000000000005749","DOIUrl":null,"url":null,"abstract":"<p><strong>Purpose: </strong>Learner engagement is the energy learners exert to remain focused and motivated to learn. The Learner Engagement Instrument (LEI) was developed to measure learner engagement in a short continuing professional development (CPD) activity. The authors validated LEI scores using validity evidence of internal structure and relationships with other variables.</p><p><strong>Method: </strong>Participants attended 1 of 4 CPD courses (1 in-person, 2 online livestreamed, and 1 either in-person or livestreamed) in 2018, 2020, 2021, and 2022. Confirmatory factor analysis was used to examine model fit for several alternative structural models, separately for each course. The authors also conducted a generalizability study to estimate score reliability. Associations were evaluated between LEI scores and Continuing Medical Education Teaching Effectiveness (CMETE) scores and participant demographics. Statistical methods accounted for repeated measures by participants.</p><p><strong>Results: </strong>Four hundred fifteen unique participants attended 203 different CPD presentations and completed the LEI 11,567 times. The originally hypothesized 4-domain model of learner engagement (domains: emotional, behavioral, cognitive in-class, cognitive out-of-class) demonstrated best model fit in all 4 courses, with comparative fit index ≥ 0.99, standardized root mean square residual ≤ 0.031, and root mean square error of approximation ≤ 0.047. The reliability for overall scores and domain scores were all acceptable (50-rater G-coefficient ≥ 0.74) except for the cognitive in-class domain (50-rater G-coefficient of 0.55 to 0.66). Findings were similar for both in-person and online delivery modalities. Correlation of LEI scores with teaching effectiveness was confirmed (rho=0.58), and a small correlation was found with participant age (rho=0.19); other associations were small and not statistically significant. Using these findings, we generated a shortened 4-item instrument, the LEI Short Form.</p><p><strong>Conclusions: </strong>This study confirms a 4-domain model of learner engagement and provides validity evidence that supports using LEI scores to measure learner engagement in both in-person and livestreamed CPD activities.</p>","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":null,"pages":null},"PeriodicalIF":5.3000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Academic Medicine","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1097/ACM.0000000000005749","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/7/29 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}
引用次数: 0

Abstract

Purpose: Learner engagement is the energy learners exert to remain focused and motivated to learn. The Learner Engagement Instrument (LEI) was developed to measure learner engagement in a short continuing professional development (CPD) activity. The authors validated LEI scores using validity evidence of internal structure and relationships with other variables.

Method: Participants attended 1 of 4 CPD courses (1 in-person, 2 online livestreamed, and 1 either in-person or livestreamed) in 2018, 2020, 2021, and 2022. Confirmatory factor analysis was used to examine model fit for several alternative structural models, separately for each course. The authors also conducted a generalizability study to estimate score reliability. Associations were evaluated between LEI scores and Continuing Medical Education Teaching Effectiveness (CMETE) scores and participant demographics. Statistical methods accounted for repeated measures by participants.

Results: Four hundred fifteen unique participants attended 203 different CPD presentations and completed the LEI 11,567 times. The originally hypothesized 4-domain model of learner engagement (domains: emotional, behavioral, cognitive in-class, cognitive out-of-class) demonstrated best model fit in all 4 courses, with comparative fit index ≥ 0.99, standardized root mean square residual ≤ 0.031, and root mean square error of approximation ≤ 0.047. The reliability for overall scores and domain scores were all acceptable (50-rater G-coefficient ≥ 0.74) except for the cognitive in-class domain (50-rater G-coefficient of 0.55 to 0.66). Findings were similar for both in-person and online delivery modalities. Correlation of LEI scores with teaching effectiveness was confirmed (rho=0.58), and a small correlation was found with participant age (rho=0.19); other associations were small and not statistically significant. Using these findings, we generated a shortened 4-item instrument, the LEI Short Form.

Conclusions: This study confirms a 4-domain model of learner engagement and provides validity evidence that supports using LEI scores to measure learner engagement in both in-person and livestreamed CPD activities.

持续专业发展学习者参与工具的验证。
目的:学习者投入度是指学习者为保持学习的专注性和积极性而付出的精力。学习者参与度工具(LEI)是为了测量学习者在短期持续专业发展(CPD)活动中的参与度而开发的。作者利用内部结构的有效性证据以及与其他变量的关系对 LEI 分数进行了验证:参与者参加了 2018 年、2020 年、2021 年和 2022 年的 4 门 CPD 课程(1 门面授课程、2 门在线直播课程和 1 门面授或在线课程)中的 1 门。作者分别对每门课程进行了确认性因子分析,以检验几种可供选择的结构模型的模型拟合度。作者还进行了一项可推广性研究,以估算分数的可靠性。还评估了 LEI 分数与继续医学教育教学效果 (CMETE) 分数和学员人口统计学之间的关联。所有统计方法都考虑了参与者的重复测量:415名参与者参加了203场不同的继续医学教育讲座,完成了11,567次LEI。最初假设的学习者参与度 4 领域模型(领域:情感、行为、课内认知、课外认知)在所有 4 门课程中均表现出最佳模型拟合度,比较拟合指数≥0.99,标准化均方根残差≤0.031,均方根近似误差≤0.047。除课堂认知领域(50 人 G 系数为 0.55 至 0.66)外,总分和领域分的信度均可接受(50 人 G 系数≥ 0.74)。所有结果在面授和在线授课模式下都相似。LEI 分数与教学效果的相关性得到了证实(rho 0.58),与参与者年龄的相关性较小(rho 0.19);其他相关性较小,且无统计学意义。根据这些研究结果,我们制作了一个简短的 4 个项目的工具,即 LEI 简表:本研究证实了学习者参与度的 4 领域模型,并提供了有效性证据,支持使用 LEI 分数来衡量学习者在现场和直播 CPD 活动中的参与度。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Academic Medicine
Academic Medicine 医学-卫生保健
CiteScore
7.80
自引率
9.50%
发文量
982
审稿时长
3-6 weeks
期刊介绍: Academic Medicine, the official peer-reviewed journal of the Association of American Medical Colleges, acts as an international forum for exchanging ideas, information, and strategies to address the significant challenges in academic medicine. The journal covers areas such as research, education, clinical care, community collaboration, and leadership, with a commitment to serving the public interest.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信