IAEAgent: LLM-Based Educational Agent for Instructional Alignment Evaluation in University Courses

Abstract

In university courses, instructional alignment evaluation reflects the degree of alignment between the intended teaching requirements and the actual content delivered by teachers. It can not only provide university administrative departments with evidence about actual teaching performance but also facilitate teachers’ targeted reflection and continuous instructional improvement. However, there is still a lack of a theory-grounded, course-aware, and quantitatively interpretable instructional alignment evaluator. To address this issue, in this article, we propose a new evaluation paradigm—a large language model (LLM)-based educational agent (IAEAgent)—for instructional alignment evaluation in university courses. First, by injecting teaching materials such as textbooks using parameter-efficient fine-tuning, a course-specific LLM with in-depth disciplinary comprehension is constructed. Then, based on this LLM and employing prompt engineering, the intended and actual instructional knowledge graphs (KGs) can be, respectively, extracted from teaching requirement documents and teachers’ actual audio transcripts of a lesson. Subsequently, guided by teaching quality evaluation theories, a quantitative analysis of instructional alignment is conducted for these two KGs across three crucial dimensions: knowledge coverage completeness, knowledge depth consistency, and knowledge structure coherence. Finally, based on the obtained analytical results, a teaching quality evaluation report containing teaching diagnostics and improvement suggestions is generated. Extensive experiments demonstrate that the IAEAgent excels in comprehensive and fine-grained instructional alignment evaluation. Furthermore, ablation studies validate the effectiveness of each step within the IAEAgent.