Towards effective teaching assistants: From intent-based chatbots to LLM-powered teaching assistants

Abstract

As chatbot technology undergoes a transformative phase in the era of artificial intelligence (AI), the integration of advanced AI models emerges as a focal point for reshaping conversational agents within the education sector. This paper explores the evolution of educational chatbot development, specifically focusing on building a teaching assistant for Data Mining and Text Analytics courses at the University of Leeds. The primary objective is to investigate and compare traditional intent-based chatbot approaches with the advanced retrieval-augmented generation (RAG) method, aiming to improve the efficiency and adaptability of teaching assistants in higher education. The study begins with the development of an Amazon Alexa teaching skill, assessing the efficacy of traditional chatbot development in higher education. To enrich the chatbot knowledge base, the research then employs an automated question–answer generation (QAG) approach using the QG Lumos Learning tool to extract contextually grounded question–answer datasets from course materials. Subsequently, the RAG-based system is proposed, leveraging LangChain with the OpenAI GPT-3.5 Turbo model. Findings highlight limitations in intent-based approaches, emphasising the need for more adaptive solutions. The proposed RAG-based teaching assistant demonstrates significant improvements in efficiently handling diverse queries, representing a paradigm shift in educational chatbot capabilities. These findings provide an in-depth understanding of the development phase, specifically illustrating the impact on chatbot performance by contrasting traditional methods with large language model-based approaches. The study contributes valuable perspectives on enhancing adaptability and effectiveness in AI-powered educational tools, providing essential considerations for future developments in the field.