The impact of GenAI on learning outcomes: A systematic review and meta-analysis of experimental studies

Abstract

Generative Artificial Intelligence (GenAI) tools such as ChatGPT, Claude, and Gemini are increasingly being integrated into educational environments, prompting questions about their actual impact on student learning. While a growing body of literature reports anecdotal or correlational evidence of GenAI's educational potential, rigorous causal evaluations remain limited. To bridge this gap, this study conducted a systematic review and meta-analysis of experimental and quasi-experimental studies investigating the effect of GenAI on learning outcomes. Following PRISMA guidelines, we screened five academic databases and identified 68 relevant studies published between 2022 and 2025. These studies yielded a total of 337 effect sizes across various educational levels, subject domains, and instructional contexts. The meta-analysis revealed a moderate overall positive effect of GenAI on learning outcomes (SMD = 0.45, 95 % CI [0.43, 0.47]), suggesting that GenAI-supported interventions are generally more effective than traditional instruction. However, substantial heterogeneity was observed across studies (I² = 95 %), indicating that the magnitude of GenAI's impact varies significantly depending on contextual and methodological factors. Moderator analyses revealed stronger effects in primary and secondary education, within natural science disciplines, and in short-term interventions with smaller sample sizes. These patterns point to both the promise and the complexity of GenAI integration in educational practice. In conclusion, while GenAI shows considerable promise for enhancing learning outcomes, its true potential will only be realized through sustained, efforts to evaluate how, when, and for whom these technologies work best in diverse learning ecosystems.