Enhancing self-directed learning and Python mastery through integration of a large language model and learning analytics dashboard

Abstract

Self-directed learning (SDL) is a critical skill in the 21st century, particularly in online Python learning environments. Learning analytics (LA) can track and analyse learning processes, which can be leveraged to prompt students to reflect on their learning strategies and progress through learning analytics dashboards (LADs). However, LADs lack pedagogical domain knowledge and fail to provide effective personalised feedback and guidance. This study designs and presents a Generative AI-powered SDL tool, SDLChat. It integrates a large language model (ERNIE-3.5) with retrieval-augmented generation (RAG) technology to generate contextualised, actionable feedback for learners across the entire SDL cycle: planning, self-monitoring and self-reflection. To evaluate the impact of SDLChat on learners' SDL skills and Python knowledge, a randomised experimental study was conducted over a six-week Python online course. The study compared the changes in SDL skills and Python knowledge of students using both SDLChat and LAD group (n = 39) and LAD-only group (n = 35). The results indicate that: (1) students using SDLChat and LAD significantly outperformed those using LAD alone in Python knowledge mastery, self-monitoring and interpersonal skills and (2) the LAD-only group showed significant improvement only in Python knowledge mastery; however, (3) no significant differences were found in posttask motivation between these two groups. This study highlights the potential of integrating LLM with learning analytics to enhance SDL skills and learning performance in online learning contexts. It also establishes a theory-informed operational framework for understanding the LLM-empowered SDL process.

Practitioner notes

What is already known about this topic

• Self-directed learning (SDL) is essential for success in online learning environments, requiring learners to plan, manage, monitor and reflect on their learning processes.
• Learning analytics (LA), particularly in the form of learning analytics dashboards (LADs), is commonly used to track SDL processes and encourage learner reflection.
• Traditional LADs are incapable of providing personalised feedback, limiting their effectiveness in enhancing SDL skills and learning performance.

What this paper adds

• Introduces SDLChat, an LLM-powered SDL tool combining a large language model (ERNIE-3.5) and retrieval-augmented generation (RAG) technology to generate contextualised and actionable feedback across the full SDL cycle.
• Provides empirical evidence from a quasi-experimental study demonstrating that the integration of SDLChat and a LAD enhances self-monitoring and interpersonal skills.
• Highlights the superiority of the integration of SDLChat and LAD in improving learning performance.
• Proposes an AI4SDL operational framework by including a technological dimension to extend SDL theory in online learning environments.

Implications for practice and/or policy

• Educators and instructional designers can leverage AI-powered tools like SDLChat to provide personalised feedback, fostering key SDL skills and improving learning outcomes in online environments.
• Policymakers should establish SDL skills as curricular objectives and implement professional development programmes to enhance teachers' digital literacy and their capacity for human–AI collaborative instruction.
• Institutions offering online courses may benefit from adopting AI-driven solutions to enhance student engagement, self-monitoring and academic performance, potentially improving course completion rates and learner satisfaction.