A meta-analysis on the effects of using AI chatbots in Korean English education

Abstract

This study aims to investigate the effects of using AI chatbots in Korean English education from a macro perspective. For this purpose, 19 experimental studies are selected to conduct a meta-analysis, synthesizing the results of 51 individual study cases. The results of this study are as follows: First, it is found that the overall effect size of using chatbots is more than medium size meaning that a chatbot is an effective tool to learn English. Second, in the aspects of linguistic competence and affective categories, each shows over medium sizes like the overall effect size. In details of the dependent variables, vocabulary and speaking in linguistic competence and motivation in affective categories, large effect sizes are shown. Third, the effect sizes are getting larger, as the younger the students are, the longer the experiment period lasts, and the more purpose-built the chatbot is. But the differences in the effect sizes in terms of these moderators (e.g., school level, experiment period, and chatbot type) are not significant. Lastly, it is suggested that follow-up studies are needed to collect a sufficient number of experimental study cases and subdivide the variables for performing a more detailed meta-anlysis.