Enhancing legal writing skills: The impact of formative feedback in a hybrid intelligence learning environment

Recent developments in artificial intelligence (AI) have significantly influenced educational technologies, reshaping the teaching and learning landscape. However, the notion of fully automating the teaching process remains contentious. This paper explores the concept of hybrid intelligence (HI), which emphasizes the synergistic collaboration between AI and humans to optimize learning outcomes. Despite the potential of AI-enhanced learning systems, their application in a human-AI collaboration system often fails to meet anticipated standards, and there needs to be more empirical evidence showcasing their effectiveness. To address this gap, this study investigates whether formative feedback in an HI learning environment helps law students learn from their errors and write more structured and persuasive legal texts. We conducted a field experiment in a law course to analyse the impact of formative feedback on the exam results of 43 law students, as well as on the writer (students), the writing product and the writing process. In the control group, students received feedback conforming to the legal common practice, where they solved legal problems and subsequently received general feedback from a lecturer based on a sample solution. Students in the treatment group were provided with formative feedback that specifically targeted their individual errors, thereby stimulating internal cognitive processes within the students. Our investigation revealed that participants who were provided with formative feedback rooted in their errors within structured and persuasive legal writing outperformed the control group in producing qualitative, better legal text during an exam. Furthermore, the analysed qualitative student statements also suggest that formative feedback promotes students' self-efficacy and self-regulated learning. Our findings indicate that integrating formative feedback rooted in individual errors enhances students' legal writing skills. This underscores the hybrid nature of AI, empowering students to identify their errors and improve in a more self-regulated manner.

Practitioner notes

What is already known about this topic

• Collaboration between humans and AI in educational settings advances learning mutually, fostering a unified developmental process.
• Collaborative education models advocate leveraging human and AI strengths for adaptive learning.
• Despite abundant theoretical research, empirical studies in HI remain limited. This gap underscores the need for more evidence-based approaches in integrating AI into educational settings.

What this paper adds

• Field experiment investigating the impact of formative feedback in a hybrid intelligence learning environment based on the theory of learning from errors.
• Comparison of a traditional legal learning environment (lecturer teaching using sample solutions) versus formative feedback in a hybrid intelligence learning environment.
• Implementing formative machine learning-based feedback supports law students in producing more structured and persuasive legal texts, leading to enhanced exam performance and higher grades.

Implications for practice and/or policy

• Our research contributes significantly to computer-based education by presenting empirical evidence of how formative writing feedback impacts students' legal knowledge and skills in educational settings. This underscores the importance of incorporating empirical data into the development of AI-based educational tools to ensure their effectiveness.
• By focusing on individual errors corrected by formative feedback, we contribute to the learning from errors literature stream. This perspective offers valuable insights into how such feedback can support students' writing and learning processes, filling a gap in empirical evidence.
• Our findings demonstrate the potential impact of ML-based learning systems, particularly in large-scale learning environments like legal mass lectures. Formative writing feedback emerges as a scalable and beneficial addition to traditional learning environments, triggering internal learning processes, fostering self-regulated learning and increasing self-efficacy among students.
• By demonstrating the effectiveness of formative feedback within the framework of HI, particularly in legal education, our research underscores the potential of combining human understanding with AI-supported feedback to enhance learning outcomes.