Empowering Instructors With AI: Evaluating the Impact of an AI-Driven Feedback Tool in Learning Analytics

Abstract

Providing timely and personalized feedback on open-ended student responses is a challenge in education due to the increased workloads and time constraints educators face. While existing research has explored how learning analytic approaches can support feedback provision, previous studies have not sufficiently investigated educators' perspectives of how these strategies affect the assessment process. This article reports on the findings of a study that aimed to evaluate the impact of an artificial intelligence (AI)-driven platform designed to assist educators in the assessment and feedback process. Leveraging large language models and learning analytics, the platform supports educators by offering tag-based recommendations and AI-generated feedback to enhance the quality and efficiency of open-response evaluations. A controlled experiment involving 65 higher education instructors assessed the platform's effectiveness in real-world environments. Using the technology acceptance model, this study investigated the platform's usefulness and relevance from the instructors' perspectives. Moreover, we collected data from the platform's usage to identify partners in instructors' behavior for different scenarios. Results indicate that AI-driven feedback significantly improved instructors' ability to provide detailed personalized feedback in less time. This study contributes to the growing research on AI applications in educational assessment and highlights key considerations for adopting AI-driven tools in instructional settings.