Harnessing GenAI for Higher Education: A Study of a Retrieval Augmented Generation Chatbot’s Impact on Learning

Abstract

Generative artificial intelligence (GenAI) and large language models (LLMs) have simultaneously opened new avenues for enhancing human learning and increased the prevalence of poor-quality information in student responses. This study introduces Professor Leodar, a custom-built, Singlish-speaking Retrieval Augmented Generation (RAG) chatbot designed to enhance materials science education by teaching computational data science skills that are essential for modern chemical research. Professor Leodar supports students in developing proficiency with statistical analysis of chemical data sets, correlation studies of materials engineering properties, and machine learning approaches to materials optimization, competencies increasingly required in contemporary chemistry practice. Deployed at Nanyang Technological University, Singapore, Professor Leodar offers personalized guidance, 24/7 availability, and contextually relevant information for materials science students transitioning from qualitative to quantitative analytical approaches. Through a mixed-methods approach, we examine the impact of Professor Leodar on learning, engagement, and exam preparedness, with 97.1% of participants reporting positive experiences. These findings help define possible roles of AI in materials science education and highlight the potential of custom GenAI chatbots for developing computational literacy in chemical sciences. Our combination of chatbot development, in-class deployment, and outcomes study offers a benchmark for GenAI educational tools addressing the field’s call for data science integration in undergraduate chemistry curricula.