Human–AI collaborative learning in mixed reality: Examining the cognitive and socio-emotional interactions

IF 8.1 1区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Belle Dang, Luna Huynh, Faaiz Gul, Carolyn Rosé, Sanna Järvelä, Andy Nguyen
{"title":"Human–AI collaborative learning in mixed reality: Examining the cognitive and socio-emotional interactions","authors":"Belle Dang,&nbsp;Luna Huynh,&nbsp;Faaiz Gul,&nbsp;Carolyn Rosé,&nbsp;Sanna Järvelä,&nbsp;Andy Nguyen","doi":"10.1111/bjet.13607","DOIUrl":null,"url":null,"abstract":"<div>\n \n <section>\n \n <p>The rise of generative artificial intelligence (GAI), especially with multimodal large language models like GPT-4o, sparked transformative potential and challenges for learning and teaching. With potential as a cognitive offloading tool, GAI can enable learners to focus on higher-order thinking and creativity. Yet, this also raises questions about integration into traditional education due to the limited research on learners' interactions with GAI. Some studies with GAI focus on text-based human–AI interactions, while research on embodied GAI in immersive environments like mixed reality (MR) remains unexplored. To address this, this study investigates interaction dynamics between learners and embodied GAI agents in MR, examining cognitive and socio-emotional interactions during collaborative learning. We investigated the paired interactive patterns between a student and an embodied GAI agent in MR, based on data from 26 higher education students with 1317 recorded activities. Data were analysed using a multi-layered learning analytics approach, including quantitative content analysis, sequence analysis via hierarchical clustering and pattern analysis through ordered network analysis (ONA). Our findings identified two interaction patterns: type (1) AI-led Supported Exploratory Questioning (AISQ) and type (2) Learner-Initiated Inquiry (LII) group. Despite their distinction in characteristic, both types demonstrated comparable levels of socio-emotional engagement and exhibited meaningful cognitive engagement, surpassing the superficial content reproduction that can be observed in interactions with GPT models. This study contributes to the human–AI collaboration and learning studies, extending understanding to learning in MR environments and highlighting implications for designing AI-based educational tools.</p>\n </section>\n \n <section>\n \n <div>\n \n <div>\n \n <h3>Practitioner notes</h3>\n <p>What is already known about this topic\n\n </p><ul>\n \n <li>Socio-emotional interactions are fundamental to cognitive processes and play a critical role in collaborative learning.</li>\n \n <li>Generative artificial intelligence (GAI) holds transformative potential for education but raises questions about how learners interact with such technology.</li>\n \n <li>Most existing research focuses on text-based interactions with GAI; there is limited empirical evidence on how embodied GAI agents within immersive environments like Mixed Reality (MR) influence the cognitive and socio-emotional interactions for learning and regulation.</li>\n </ul>\n <p>What this paper adds\n\n </p><ul>\n \n <li>Provides first empirical insights into cognitive and socio-emotional interaction patterns between learners and embodied GAI agents in MR environments.</li>\n \n <li>Identifies two distinct interaction patterns: AISQ type (structured, guided, supportive) and LII type (inquiry-driven, exploratory, engaging), demonstrating how these patterns influence collaborative learning dynamics.</li>\n \n <li>Shows that both interaction types facilitate meaningful cognitive engagement, moving beyond superficial content reproduction commonly associated with GAI interactions.</li>\n </ul>\n <p>Implications for practice and/or policy\n\n </p><ul>\n \n <li>Insights from the identified interaction patterns can inform the design of teaching strategies that effectively integrate embodied GAI agents to enhance both cognitive and socio-emotional engagement.</li>\n \n <li>Findings can guide the development of AI-based educational tools that capitalise on the capabilities of embodied GAI agents, supporting a balance between structured guidance and exploratory learning.</li>\n \n <li>Highlights the need for ethical considerations in adopting embodied GAI agents, particularly regarding the human-like realism of these agents and potential impacts on learner dependency and interaction norms.</li>\n </ul>\n </div>\n </div>\n </section>\n </div>","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":"56 5","pages":"2078-2101"},"PeriodicalIF":8.1000,"publicationDate":"2025-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://bera-journals.onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13607","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Journal of Educational Technology","FirstCategoryId":"95","ListUrlMain":"https://bera-journals.onlinelibrary.wiley.com/doi/10.1111/bjet.13607","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

The rise of generative artificial intelligence (GAI), especially with multimodal large language models like GPT-4o, sparked transformative potential and challenges for learning and teaching. With potential as a cognitive offloading tool, GAI can enable learners to focus on higher-order thinking and creativity. Yet, this also raises questions about integration into traditional education due to the limited research on learners' interactions with GAI. Some studies with GAI focus on text-based human–AI interactions, while research on embodied GAI in immersive environments like mixed reality (MR) remains unexplored. To address this, this study investigates interaction dynamics between learners and embodied GAI agents in MR, examining cognitive and socio-emotional interactions during collaborative learning. We investigated the paired interactive patterns between a student and an embodied GAI agent in MR, based on data from 26 higher education students with 1317 recorded activities. Data were analysed using a multi-layered learning analytics approach, including quantitative content analysis, sequence analysis via hierarchical clustering and pattern analysis through ordered network analysis (ONA). Our findings identified two interaction patterns: type (1) AI-led Supported Exploratory Questioning (AISQ) and type (2) Learner-Initiated Inquiry (LII) group. Despite their distinction in characteristic, both types demonstrated comparable levels of socio-emotional engagement and exhibited meaningful cognitive engagement, surpassing the superficial content reproduction that can be observed in interactions with GPT models. This study contributes to the human–AI collaboration and learning studies, extending understanding to learning in MR environments and highlighting implications for designing AI-based educational tools.

Practitioner notes

What is already known about this topic

  • Socio-emotional interactions are fundamental to cognitive processes and play a critical role in collaborative learning.
  • Generative artificial intelligence (GAI) holds transformative potential for education but raises questions about how learners interact with such technology.
  • Most existing research focuses on text-based interactions with GAI; there is limited empirical evidence on how embodied GAI agents within immersive environments like Mixed Reality (MR) influence the cognitive and socio-emotional interactions for learning and regulation.

What this paper adds

  • Provides first empirical insights into cognitive and socio-emotional interaction patterns between learners and embodied GAI agents in MR environments.
  • Identifies two distinct interaction patterns: AISQ type (structured, guided, supportive) and LII type (inquiry-driven, exploratory, engaging), demonstrating how these patterns influence collaborative learning dynamics.
  • Shows that both interaction types facilitate meaningful cognitive engagement, moving beyond superficial content reproduction commonly associated with GAI interactions.

Implications for practice and/or policy

  • Insights from the identified interaction patterns can inform the design of teaching strategies that effectively integrate embodied GAI agents to enhance both cognitive and socio-emotional engagement.
  • Findings can guide the development of AI-based educational tools that capitalise on the capabilities of embodied GAI agents, supporting a balance between structured guidance and exploratory learning.
  • Highlights the need for ethical considerations in adopting embodied GAI agents, particularly regarding the human-like realism of these agents and potential impacts on learner dependency and interaction norms.

Abstract Image

Abstract Image

Abstract Image

Abstract Image

混合现实中的人类-人工智能协作学习:研究认知和社会情感互动
生成式人工智能(GAI)的兴起,尤其是像gpt - 40这样的多模态大型语言模型,激发了学习和教学的变革潜力和挑战。作为一种潜在的认知卸载工具,GAI可以使学习者专注于高阶思维和创造力。然而,由于对学习者与GAI互动的研究有限,这也提出了融入传统教育的问题。一些关于GAI的研究主要集中在基于文本的人类与ai交互上,而对混合现实(MR)等沉浸式环境中体现的GAI的研究仍未得到探索。为了解决这个问题,本研究调查了MR中学习者和具体GAI代理之间的互动动态,研究了协作学习过程中的认知和社会情感互动。基于26名高等教育学生1317次记录活动的数据,我们在MR中调查了学生和具身GAI代理之间的配对互动模式。数据分析采用多层学习分析方法,包括定量内容分析、层次聚类的序列分析和有序网络分析(ONA)的模式分析。我们的发现确定了两种互动模式:类型(1)人工智能主导的支持探索性提问(AISQ)和类型(2)学习者发起的提问(LII)组。尽管它们的特征不同,但这两种类型都表现出相当水平的社会情感参与,并表现出有意义的认知参与,超越了在与GPT模型交互中可以观察到的肤浅内容复制。本研究有助于人类与人工智能的协作和学习研究,将理解扩展到MR环境中的学习,并强调了设计基于人工智能的教育工具的意义。社会情感互动是认知过程的基础,在协作学习中起着至关重要的作用。生成式人工智能(GAI)对教育具有变革潜力,但也提出了学习者如何与这种技术互动的问题。大多数现有的研究都集中在基于文本的GAI交互;在像混合现实(MR)这样的沉浸式环境中,具体的GAI代理如何影响学习和调节的认知和社会情感互动,经验证据有限。本文提供了MR环境中学习者和具身GAI代理之间的认知和社会情感互动模式的第一个经验见解。确定了两种不同的交互模式:AISQ类型(结构化、引导、支持)和LII类型(探究驱动、探索、参与),展示了这些模式如何影响协作学习动态。这两种互动类型都促进了有意义的认知参与,超越了通常与GAI互动相关的肤浅内容复制。对实践和/或政策的启示从确定的互动模式的见解可以告知教学策略的设计,有效地整合具身GAI代理,以提高认知和社会情感参与。研究结果可以指导基于人工智能的教育工具的开发,这些工具可以利用嵌入的人工智能代理的能力,支持结构化指导和探索性学习之间的平衡。强调在采用具身GAI代理时需要考虑伦理因素,特别是这些代理的类人现实主义以及对学习者依赖和交互规范的潜在影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
British Journal of Educational Technology
British Journal of Educational Technology EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
15.60
自引率
4.50%
发文量
111
期刊介绍: BJET is a primary source for academics and professionals in the fields of digital educational and training technology throughout the world. The Journal is published by Wiley on behalf of The British Educational Research Association (BERA). It publishes theoretical perspectives, methodological developments and high quality empirical research that demonstrate whether and how applications of instructional/educational technology systems, networks, tools and resources lead to improvements in formal and non-formal education at all levels, from early years through to higher, technical and vocational education, professional development and corporate training.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信