Human-AI collaboration: Designing artificial agents to facilitate socially shared regulation among learners

Socially shared regulation of learning (SSRL) is a crucial process for groups of learners to successfully collaborate. Detecting and supporting SSRL is a challenge, especially in real time, but hybrid intelligence approaches such as Artificial Intelligence (AI) agents may make this possible. Leveraging the concept of trigger events which invite SSRL, we present a design of an AI agent, MAI, which can detect SSRL and prompt students to raise their group-level metacognitive awareness with the aim of facilitating SSRL. We present the methodology we used to design MAI, drawing on the Echeloned DSR (eDSR) Methodological Framework and making use of the Wizard of Oz prototyping paradigm. We likewise present empirical results evaluating our initial prototype of MAI, using lexical alignment between speakers as a quantitative measure of the effect of MAI's prompts on facilitating SSRL, the Partner Model Questionnaire as a quantitative measure of perceptions of MAI, and interviews as qualitative context for these perceptions. We found that the first prototype of MAI did not facilitate SSRL as hoped, possibly owing to mixed perceptions of MAI's reliability and lack of clarity about MAI's role in the collaborative learning task. From these findings, we offer revised prompts for the next iteration of prototyping this agent and a refined set of design requirements for future development of metacognitive AI agents for supporting SSRL.

Practitioner notes

What is already known about this topic

• Socially Shared Regulation of Learning (SSRL) is recognized as a critical component for the success of collaborative learning, emphasizing the importance of group-level regulatory processes in achieving shared goals, enacting strategies and monitoring learning progress.
• Supporting SSRL in face-to-face collaborative learning environments presents challenges, including the complexity of coordinating and synchronizing individual contributions and regulatory actions within a group context.

What this paper adds

• This paper introduces the design of Metacognitive Artificial Intelligence (MAI), a novel AI system aimed at enhancing Human-AI collaboration for supporting and augmenting SSRL processes.
• Through empirical research, the study offers lessons learned and design considerations for developing artificial agents on facilitating and enhancing SSRL among learners, demonstrating how AI can play a pivotal role in collaborative learning environments.
• The findings highlight the critical importance of multidisciplinary knowledge in the design of multi-agent interfaces (MAI) that provide real-time, adaptive support for group metacognitive processes and decision-making.

Implications for practice and/or policy

• Educational technologists can utilize the proposed design principles in the development and integration of MAI tools to enhance SSRL.
• Educators can incorporate the principles of MAI and our relevant findings into their teaching strategies to actively foster and support socially shared regulation of learning among students.
• Policymakers should consider revising educational frameworks to include the use of AI technologies that support SSRL strategies in collaborative learning.