Impact of ML-LA feedback system on learners’ academic performance, engagement and behavioral patterns in online collaborative learning environments: A lag sequential analysis and Markov chain approach

IF 4.8 2区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Hatice Yildiz Durak
{"title":"Impact of ML-LA feedback system on learners’ academic performance, engagement and behavioral patterns in online collaborative learning environments: A lag sequential analysis and Markov chain approach","authors":"Hatice Yildiz Durak","doi":"10.1007/s10639-024-12911-9","DOIUrl":null,"url":null,"abstract":"<p>Feedback is critical in providing personalized information about educational processes and supporting their performance in online collaborative learning environments. However, giving effective feedback and monitoring its effects, which is especially important in online environments, is a complex issue. Although providing feedback by analyzing online learning behaviors, it is unclear how the effectiveness of this feedback translates into online learning experiences. The current study aims to compare the behavioral patterns of online system engagement of students who receive and do not receive machine learning-based temporal learning analytics (ML-LA) feedback, to identify the differences between student groups in terms of learning performance, online engagement, and various system usage variables, and to examine the behavioral patterns change over time of students regarding online system engagement. The current study was conducted with the participation of 49 undergraduate students. The study defined three engagement levels using system usage analytics and cluster analysis. While t-test and ANCOVA were applied to pre-test and post-test scores to evaluate students’ learning performance and online engagement, lag sequential analysis was used to analyze behavioral patterns, and the Markov chain was used to examine the change of behavioral patterns over time. The group receiving ML-LA feedback showed higher behavior and cognitive engagement than the control group. In addition, the rate of completing learning tasks was higher in the experimental group. Temporal patterns of online engagement behaviors across student groups are described and compared. The results showed that both groups used all stages of the system features. However, there were some differences in the navigation rankings. The most important behavioral transitions in the experimental group are task and discussion viewing and posting, task posting updating, and group performance viewing. In the control group, the most important behavioral transitions are the relationship between viewing a discussion and making a discussion, then this is followed by the sequential relationship between viewing individual performance and viewing group performance. The results showed that students’ engagement behaviors transitioned from light to medium and intense throughout the semester, especially in the experimental group. For learning designers and researchers, this study can help develop a deep understanding of environment design.</p>","PeriodicalId":51494,"journal":{"name":"Education and Information Technologies","volume":null,"pages":null},"PeriodicalIF":4.8000,"publicationDate":"2024-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Education and Information Technologies","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1007/s10639-024-12911-9","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

Feedback is critical in providing personalized information about educational processes and supporting their performance in online collaborative learning environments. However, giving effective feedback and monitoring its effects, which is especially important in online environments, is a complex issue. Although providing feedback by analyzing online learning behaviors, it is unclear how the effectiveness of this feedback translates into online learning experiences. The current study aims to compare the behavioral patterns of online system engagement of students who receive and do not receive machine learning-based temporal learning analytics (ML-LA) feedback, to identify the differences between student groups in terms of learning performance, online engagement, and various system usage variables, and to examine the behavioral patterns change over time of students regarding online system engagement. The current study was conducted with the participation of 49 undergraduate students. The study defined three engagement levels using system usage analytics and cluster analysis. While t-test and ANCOVA were applied to pre-test and post-test scores to evaluate students’ learning performance and online engagement, lag sequential analysis was used to analyze behavioral patterns, and the Markov chain was used to examine the change of behavioral patterns over time. The group receiving ML-LA feedback showed higher behavior and cognitive engagement than the control group. In addition, the rate of completing learning tasks was higher in the experimental group. Temporal patterns of online engagement behaviors across student groups are described and compared. The results showed that both groups used all stages of the system features. However, there were some differences in the navigation rankings. The most important behavioral transitions in the experimental group are task and discussion viewing and posting, task posting updating, and group performance viewing. In the control group, the most important behavioral transitions are the relationship between viewing a discussion and making a discussion, then this is followed by the sequential relationship between viewing individual performance and viewing group performance. The results showed that students’ engagement behaviors transitioned from light to medium and intense throughout the semester, especially in the experimental group. For learning designers and researchers, this study can help develop a deep understanding of environment design.

Abstract Image

ML-LA 反馈系统对在线协作学习环境中学习者学习成绩、参与度和行为模式的影响:滞后序列分析和马尔可夫链方法
在在线协作学习环境中,反馈对于提供有关教育过程的个性化信息和支持他们的表现至关重要。然而,如何提供有效的反馈并监控其效果(这在在线环境中尤为重要)是一个复杂的问题。虽然通过分析在线学习行为来提供反馈,但目前还不清楚这种反馈的有效性如何转化为在线学习体验。本研究旨在比较接受和未接受基于机器学习的时态学习分析(ML-LA)反馈的学生参与在线系统的行为模式,确定学生群体在学习成绩、在线参与度和各种系统使用变量方面的差异,并考察学生参与在线系统的行为模式随时间的变化。本研究有 49 名本科生参与。研究利用系统使用分析和聚类分析定义了三个参与度等级。研究采用了 t 检验和方差分析来评估学生的学习成绩和在线参与度,采用了滞后序列分析来分析行为模式,采用了马尔可夫链来研究行为模式随时间的变化。与对照组相比,接受 ML-LA 反馈的小组表现出更高的行为和认知参与度。此外,实验组的学习任务完成率也更高。研究还描述并比较了各组学生在线参与行为的时间模式。结果显示,两组学生都使用了系统所有阶段的功能。不过,在导航排名方面存在一些差异。实验组最重要的行为转换是任务和讨论的查看和发布、任务发布的更新以及小组表现的查看。在对照组中,最重要的行为转换是查看讨论和进行讨论之间的关系,然后是查看个人表现和查看小组表现之间的顺序关系。结果显示,在整个学期中,学生的参与行为从轻度过渡到中度和重度,尤其是在实验组中。对于学习设计者和研究人员来说,这项研究有助于深入理解环境设计。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Education and Information Technologies
Education and Information Technologies EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
10.00
自引率
12.70%
发文量
610
期刊介绍: The Journal of Education and Information Technologies (EAIT) is a platform for the range of debates and issues in the field of Computing Education as well as the many uses of information and communication technology (ICT) across many educational subjects and sectors. It probes the use of computing to improve education and learning in a variety of settings, platforms and environments. The journal aims to provide perspectives at all levels, from the micro level of specific pedagogical approaches in Computing Education and applications or instances of use in classrooms, to macro concerns of national policies and major projects; from pre-school classes to adults in tertiary institutions; from teachers and administrators to researchers and designers; from institutions to online and lifelong learning. The journal is embedded in the research and practice of professionals within the contemporary global context and its breadth and scope encourage debate on fundamental issues at all levels and from different research paradigms and learning theories. The journal does not proselytize on behalf of the technologies (whether they be mobile, desktop, interactive, virtual, games-based or learning management systems) but rather provokes debate on all the complex relationships within and between computing and education, whether they are in informal or formal settings. It probes state of the art technologies in Computing Education and it also considers the design and evaluation of digital educational artefacts.  The journal aims to maintain and expand its international standing by careful selection on merit of the papers submitted, thus providing a credible ongoing forum for debate and scholarly discourse. Special Issues are occasionally published to cover particular issues in depth. EAIT invites readers to submit papers that draw inferences, probe theory and create new knowledge that informs practice, policy and scholarship. Readers are also invited to comment and reflect upon the argument and opinions published. EAIT is the official journal of the Technical Committee on Education of the International Federation for Information Processing (IFIP) in partnership with UNESCO.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信