Metacognition meets AI: Empowering reflective writing with large language models

Reflective writing is known as a useful method in learning sciences to improve the metacognitive skills of students. However, students struggle to structure their reflections properly, limiting the possible learning gains. Previous works in educational technologies literature have explored the paradigms of learning from worked and modelling examples, but (a) their application to the domain of reflective writing is rare, (b) such methods might not scale properly to large-scale classrooms, and (c) they do not necessarily take the learning needs of each student into account. In this work, we suggest two approaches of integrating AI-enabled support in digital systems designed around learning from worked and modelling examples paradigms, to provide personalized learning and feedback to students using large language models (LLMs). We evaluate Reflectium, our reflective writing assistant, show benefits of integrating AI support into the learning from examples modalities and compare the perception of the users and their interaction behaviour when using each version of our tool. Our work sheds light on the applicability of generative LLMs to different types of providing support using the learning from examples paradigm, in the domain of reflective writing.

Practitioner notes

What is already known about this topic

• Reflective writing fosters metacognitive skills and improves learning gains and personal growth.
• The learning from worked and modelling examples paradigms is effective for skill acquisition and applying the acquired knowledge.
• Existing reflective writing assistants usually lack dynamic, AI-driven feedback or interactivity, limiting personalization and adaptability to each user's own needs in the learning process.

What this paper adds

• It introduces Reflectium, an AI-enabled reflective writing assistant, integrating intelligent and interactive writing support for both the learning from worked and modelling examples paradigms.
• It demonstrates the use of a fine-tuned large language model (LLM) for providing feedback in the learning from worked examples version, and an LLM-powered conversational agent simulating instructor interactions for the learning from modelling examples version.
• It reports findings from a user study comparing the positive impact of artificial intelligence (AI) support on learners' performance, interaction behaviour and learning experience.

Implications for practice and/or policy

• Digital tutoring systems for teaching reflective writing using the learning from worked examples paradigm should incorporate adaptive AI feedback to enhance learning gains.
• Conversational agents simulating peers/instructors and powered by LLMs can provide scalable, interactive support for learning from modelling examples, notably in large-scale educational settings.
• Reflective writing tools should be evaluated for their impact on different aspects of the learning process, such as task performance, interaction behaviour and user experience, to guide future improvements.
• Educators and policymakers should consider the integration of AI-driven reflective writing tools into teaching curricula to enhance reflective practices and metacognitive skill development.