Chatbots in education: A systematic review of objectives, underlying technology and theory, evaluation criteria, and impacts

Abstract

There is a growing body of literature on the development and use of chatbots in education. However, insights into the objectives, underlying technologies, theories, and criteria for developing and evaluating chatbots in education are lacking. This study presents a systematic review of 71 papers, identified through a thorough search in Web of Science and Scopus in October 2023, to understand, compare, and create a comprehensive overview of these elements. Papers were selected through a step-by-step procedure based on the guidelines of the PRISMA framework. The main results indicate that most chatbots in education are teaching-oriented and have increasingly been developed using chatbot builder platforms. However, many of these chatbots are integrated into educational settings without a solid theoretical foundation. Despite this, they are primarily evaluated based on perceptual factors and, in most cases, prove effective for their intended objectives. This review also identified a few limitations of the included studies, such as heterogeneity, publication bias, and focus on short-term impact. Overall, this review enhances our understanding of chatbots in education and provides valuable insight and guidance for educators, practitioners, and researchers.