University Students' Perceptions of a Multimodal AI System for Real-World Collaboration Analytics: Lessons Learned From a Case Study

IF 4.6 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2025-07-30 DOI:10.1111/jcal.70103

Wannapon Suraworachet, Qi Zhou, Mutlu Cukurova

{"title":"University Students' Perceptions of a Multimodal AI System for Real-World Collaboration Analytics: Lessons Learned From a Case Study","authors":"Wannapon Suraworachet, Qi Zhou, Mutlu Cukurova","doi":"10.1111/jcal.70103","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Many researchers work on the design and development of multimodal collaboration support systems with AI, yet very few of these systems are mature enough to provide actionable feedback to students in real-world settings. Therefore, a notable gap exists in the literature regarding students' perceptions of such systems and the feedback they generate.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>This study designed, built and implemented a set of collaboration analytics to capture, interpret and provide feedback on students' collaborative processes, including their non-verbal group interactions as well as group challenges and regulation arising from discourse in authentic collocated collaborative settings.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>Seven groups of five to six postgraduate students with varying backgrounds participated in face-to-face collaborative design tasks (<i>n</i> = 36) for an 11-week-long semester. Multimodal data from audio and video recordings of collaborative learning sessions were analysed using various machine learning techniques to model students' group processes and to generate feedback. A post hoc evaluation of the collaboration analytics feedback was conducted using individual student reflections and focus group interviews.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusions</h3>\n \n <p>The findings suggest that analytics feedback has the potential to promote students' understanding of their collaborative processes (e.g., awareness of individual, peer and group behaviours and alterations at the individual level). However, the study also identified significant limitations and challenges associated with the real-world application of collaboration analytics (e.g., limited group transactions stemmed from a lack of group interpretative sessions). The paper concludes with a discussion on future design suggestions and principles (e.g., an integration of analytics with the learning design, value alignments among stakeholders and roles of teachers).</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 5","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2025-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jcal.70103","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.70103","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Many researchers work on the design and development of multimodal collaboration support systems with AI, yet very few of these systems are mature enough to provide actionable feedback to students in real-world settings. Therefore, a notable gap exists in the literature regarding students' perceptions of such systems and the feedback they generate.

Objectives

This study designed, built and implemented a set of collaboration analytics to capture, interpret and provide feedback on students' collaborative processes, including their non-verbal group interactions as well as group challenges and regulation arising from discourse in authentic collocated collaborative settings.

Methods

Seven groups of five to six postgraduate students with varying backgrounds participated in face-to-face collaborative design tasks (n = 36) for an 11-week-long semester. Multimodal data from audio and video recordings of collaborative learning sessions were analysed using various machine learning techniques to model students' group processes and to generate feedback. A post hoc evaluation of the collaboration analytics feedback was conducted using individual student reflections and focus group interviews.

Results and Conclusions

The findings suggest that analytics feedback has the potential to promote students' understanding of their collaborative processes (e.g., awareness of individual, peer and group behaviours and alterations at the individual level). However, the study also identified significant limitations and challenges associated with the real-world application of collaboration analytics (e.g., limited group transactions stemmed from a lack of group interpretative sessions). The paper concludes with a discussion on future design suggestions and principles (e.g., an integration of analytics with the learning design, value alignments among stakeholders and roles of teachers).

Abstract Image

查看原文本刊更多论文

大学生对用于现实协作分析的多模态人工智能系统的看法：从案例研究中获得的经验教训

许多研究人员致力于设计和开发具有人工智能的多模式协作支持系统，但这些系统很少成熟到足以在现实环境中为学生提供可操作的反馈。因此，关于学生对这些系统的看法及其产生的反馈，文献中存在着显著的差距。本研究设计、构建并实施了一套协作分析，以捕捉、解释和反馈学生的协作过程，包括他们的非语言群体互动，以及在真实的协同协作环境中由话语产生的群体挑战和规则。方法7组5 ~ 6名不同背景的研究生参与面对面协作设计任务（n = 36），为期11周。使用各种机器学习技术分析来自协作学习会议的音频和视频记录的多模态数据，以模拟学生的小组过程并产生反馈。利用个别学生的反思和焦点小组访谈，对协作分析反馈进行了事后评估。结果和结论研究结果表明，分析反馈有可能促进学生对他们的合作过程的理解（例如，对个人、同伴和群体行为以及个人层面的改变的认识）。然而，该研究还确定了与协作分析的实际应用相关的重大限制和挑战（例如，由于缺乏小组解释会议，有限的小组交易）。本文最后讨论了未来的设计建议和原则（例如，分析与学习设计的整合，利益相关者之间的价值定位和教师的角色）。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope