Development and implementation of a generative AI-based personalized recommender system to improve students’ self-regulated learning and academic performance

Abstract

Students' ability to manage their own learning is crucial for academic success, but many struggle with self-regulation, often leading to disengagement. Past studies have explored tactics to boost students' self-regulated learning (SRL), such as writing self-reflective reports and incorporating prompts in video lectures. These tactics, however, often lack timely, personalized feedback due to the labor-intensive nature of such support. In this study, we report the development and implementation of an innovative large language model (LLM)-based recommender system named SRLAdvisor to address these limitations. The system combines a web interface with a series of LLM-based agents, allowing students to receive immediate, personalized feedback. By processing students' natural language interactions in real-time, SRLAdvisor dynamically profiles their self-regulation and delivers personalized recommendations. This study involved 22 postgraduate students and employed a sequential explanatory mixed-methods approach to evaluate students’ self-regulation and learning performance. We found near-perfect agreement between the system detection and human coding. SRLAdvisor exhibited greater recommendation precision in self-monitoring and self-evaluation tasks. Students who perceived the system as highly useful showed significantly greater self-reported SRL gains than those with lower perceptions. Additionally, a k-means cluster analysis indicated that students who engaged more frequently with SRLAdvisor achieved significantly greater improvements in learning performance. These findings underscore the potential of leveraging LLMs to deliver personalized recommendations. However, they also reveal potential concerns, such as over-reliance on AI and prompt monotony, that need to be further examined. These issues are particularly relevant in the context of LLM-supported SRL interventions and warrant further discussion.