The Impact of ChatGPT on Students' Academic Achievement: A Meta-Analysis

IF 4.6 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2025-07-15 DOI:10.1111/jcal.70096

Zhiwei Liu, Haode Zuo, Yongjing Lu

{"title":"The Impact of ChatGPT on Students' Academic Achievement: A Meta-Analysis","authors":"Zhiwei Liu, Haode Zuo, Yongjing Lu","doi":"10.1111/jcal.70096","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>ChatGPT, a generative artificial intelligence (GenAI) chatbot, has gained significant traction as a tool for supporting students learning. Despite its growing popularity, there is still no academic consensus on its effectiveness in enhancing students' academic achievement.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>This study aims to explore the effect of ChatGPT on students' academic achievement through a meta-analysis. It seeks to identify the overall effect size and examine variations based on moderators such as educational level, discipline, intervention duration, sample size, knowledge type, instructional model, role-setting, learning approach, and generated content.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>A meta-analysis was conducted on 37 studies (comprising 37 effect sizes) published between 2022 and 2025. The studies were analysed to calculate the overall effect size (Hedges' <i>g</i>) and to explore subgroup differences.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusion</h3>\n \n <p>The findings reveal that ChatGPT has a moderately positive impact on students' academic achievement, with an overall effect size of <i>g</i> = 0.577 (95% CI [0.395, 0.759], <i>p</i> < 0.001). Further analysis of moderating variables indicates that no significant differences are observed across educational levels, role-setting, or learning approaches. A greater effect is observed in the social sciences compared to other disciplines; an intervention duration of 5–10 weeks has a larger impact on academic achievement compared to other durations; sample sizes ranging from 21 to 40 participants exhibit a larger impact on academic achievement than other sample sizes; ChatGPT is more effective in supporting the learning of declarative knowledge compared to procedural knowledge; the combination of traditional classrooms with ChatGPT is more effective than using ChatGPT in a flipped classroom; compared to generating code, using ChatGPT to generate text has better academic achievement.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 4","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2025-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.70096","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

ChatGPT, a generative artificial intelligence (GenAI) chatbot, has gained significant traction as a tool for supporting students learning. Despite its growing popularity, there is still no academic consensus on its effectiveness in enhancing students' academic achievement.

Objectives

This study aims to explore the effect of ChatGPT on students' academic achievement through a meta-analysis. It seeks to identify the overall effect size and examine variations based on moderators such as educational level, discipline, intervention duration, sample size, knowledge type, instructional model, role-setting, learning approach, and generated content.

Methods

A meta-analysis was conducted on 37 studies (comprising 37 effect sizes) published between 2022 and 2025. The studies were analysed to calculate the overall effect size (Hedges' g) and to explore subgroup differences.

Results and Conclusion

The findings reveal that ChatGPT has a moderately positive impact on students' academic achievement, with an overall effect size of g = 0.577 (95% CI [0.395, 0.759], p < 0.001). Further analysis of moderating variables indicates that no significant differences are observed across educational levels, role-setting, or learning approaches. A greater effect is observed in the social sciences compared to other disciplines; an intervention duration of 5–10 weeks has a larger impact on academic achievement compared to other durations; sample sizes ranging from 21 to 40 participants exhibit a larger impact on academic achievement than other sample sizes; ChatGPT is more effective in supporting the learning of declarative knowledge compared to procedural knowledge; the combination of traditional classrooms with ChatGPT is more effective than using ChatGPT in a flipped classroom; compared to generating code, using ChatGPT to generate text has better academic achievement.

查看原文本刊更多论文

聊天技巧对学生学业成绩的影响：一个元分析

ChatGPT是一种生成式人工智能（GenAI）聊天机器人，作为一种支持学生学习的工具已经获得了显著的吸引力。尽管它越来越受欢迎，但它在提高学生学习成绩方面的有效性仍然没有学术共识。目的本研究旨在通过荟萃分析探讨聊天技能对学生学业成绩的影响。它试图确定总体效应大小，并检查基于调节因素的变化，如教育水平、学科、干预持续时间、样本量、知识类型、教学模式、角色设定、学习方法和生成内容。方法对2022 - 2025年间发表的37项研究（包括37个效应量）进行meta分析。对这些研究进行分析，以计算总体效应大小（Hedges' g）并探索亚组差异。结果与结论研究结果显示，ChatGPT对学生学业成绩有中等正向影响，总体效应量g = 0.577 （95% CI [0.395, 0.759], p < 0.001）。对调节变量的进一步分析表明，在教育水平、角色设置或学习方法之间没有观察到显著差异。与其他学科相比，社会科学的影响更大；5-10周的干预时间对学业成绩的影响大于其他干预时间；21至40名参与者的样本量对学业成绩的影响大于其他样本量；ChatGPT对陈述性知识的学习比程序性知识的学习更有效；传统课堂与ChatGPT的结合比在翻转课堂中使用ChatGPT更有效；与生成代码相比，使用ChatGPT生成文本具有更好的学术成就。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope