A Meta-Analysis of the Impact of Generative Artificial Intelligence on Learning Outcomes

IF 4.6 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2025-09-01 DOI:10.1111/jcal.70117

Nan Ma, Zhiyong Zhong

{"title":"A Meta-Analysis of the Impact of Generative Artificial Intelligence on Learning Outcomes","authors":"Nan Ma, Zhiyong Zhong","doi":"10.1111/jcal.70117","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>With the rapid advancement of technology, the integration of Generative Artificial Intelligence (GAI) in education has gained considerable attention. Many studies have examined GAI's impact on learning outcomes, yet their conclusions are inconsistent, highlighting the need for a comprehensive review to clarify its overall effects and identify influential factors.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>This study aims to conduct a meta-analysis of the effects of GAI on student learning outcomes across cognitive, competency and affective dimensions. Additionally, it seeks to explore how various moderating factors, including subject discipline, instructional duration, knowledge type, prior knowledge and tool type, influence GAI's effectiveness.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>A meta-analysis was performed on 34 experimental and quasi-experimental studies published internationally. Effect sizes were calculated for overall learning outcomes and categorised by dimension. Further analysis was conducted to assess the influence of moderating variables on the impact of GAI.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>The meta-analysis indicates that Generative Artificial Intelligence has a significant positive impact on overall learning outcomes, with a combined effect size of 0.68 (<i>p</i> < 0.001). The impact is particularly pronounced in the cognitive dimension (<i>g</i> = 0.795) and the competency dimension (<i>g</i> = 0.711), while its effect on the affective dimension (<i>g</i> = 0.507) is moderate but still significant. The analysis of moderating variables reveals that the effectiveness of GAI is influenced by discipline type but is not significantly affected by instructional period, knowledge type, prior knowledge level, or tool type. Specifically, GAI exhibits the highest positive effects in mathematics, science and humanities, whereas its impact is relatively lower yet still significant in computer science and medical/nursing education. Additionally, GAI's effectiveness does not significantly differ across various instructional periods, different knowledge types, learners with varying prior knowledge levels, or different AI tool versions.</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>To optimise GAI's use in education, the study suggests aligning GAI with specific subject needs, adapting tools for different student levels, integrating GAI with traditional teaching and establishing monitoring mechanisms. These strategies aim to maximise GAI's positive impact on learning efficiency and quality across educational settings.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 5","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.70117","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

With the rapid advancement of technology, the integration of Generative Artificial Intelligence (GAI) in education has gained considerable attention. Many studies have examined GAI's impact on learning outcomes, yet their conclusions are inconsistent, highlighting the need for a comprehensive review to clarify its overall effects and identify influential factors.

Objectives

This study aims to conduct a meta-analysis of the effects of GAI on student learning outcomes across cognitive, competency and affective dimensions. Additionally, it seeks to explore how various moderating factors, including subject discipline, instructional duration, knowledge type, prior knowledge and tool type, influence GAI's effectiveness.

Methods

A meta-analysis was performed on 34 experimental and quasi-experimental studies published internationally. Effect sizes were calculated for overall learning outcomes and categorised by dimension. Further analysis was conducted to assess the influence of moderating variables on the impact of GAI.

Results

The meta-analysis indicates that Generative Artificial Intelligence has a significant positive impact on overall learning outcomes, with a combined effect size of 0.68 (p < 0.001). The impact is particularly pronounced in the cognitive dimension (g = 0.795) and the competency dimension (g = 0.711), while its effect on the affective dimension (g = 0.507) is moderate but still significant. The analysis of moderating variables reveals that the effectiveness of GAI is influenced by discipline type but is not significantly affected by instructional period, knowledge type, prior knowledge level, or tool type. Specifically, GAI exhibits the highest positive effects in mathematics, science and humanities, whereas its impact is relatively lower yet still significant in computer science and medical/nursing education. Additionally, GAI's effectiveness does not significantly differ across various instructional periods, different knowledge types, learners with varying prior knowledge levels, or different AI tool versions.

Conclusions

To optimise GAI's use in education, the study suggests aligning GAI with specific subject needs, adapting tools for different student levels, integrating GAI with traditional teaching and establishing monitoring mechanisms. These strategies aim to maximise GAI's positive impact on learning efficiency and quality across educational settings.

查看原文本刊更多论文

生成式人工智能对学习结果影响的荟萃分析

随着科技的飞速发展，生成式人工智能（GAI）在教育中的应用受到了广泛的关注。许多研究调查了GAI对学习成果的影响，但他们的结论不一致，突出表明需要进行全面审查，以澄清其总体影响并确定影响因素。本研究旨在从认知、能力和情感三个维度对GAI对学生学习成果的影响进行meta分析。此外，它试图探索各种调节因素，包括学科学科，教学时间，知识类型，先验知识和工具类型，如何影响GAI的有效性。方法对国际上发表的34篇实验和准实验研究进行meta分析。计算总体学习结果的效应量，并按维度分类。进一步分析了调节变量对GAI影响的影响。荟萃分析表明，生成式人工智能对整体学习结果有显著的积极影响，综合效应值为0.68 （p < 0.001）。对认知维度（g = 0.795）和胜任力维度（g = 0.711）的影响尤为显著，对情感维度（g = 0.507）的影响虽不明显，但依然显著。调节变量分析表明，GAI的有效性受学科类型的影响，而受教学时间、知识类型、先验知识水平和工具类型的影响不显著。具体而言，GAI在数学、科学和人文学科中表现出最高的积极影响，而在计算机科学和医学/护理教育方面的影响相对较低，但仍然显著。此外，GAI的有效性在不同的教学周期、不同的知识类型、不同先验知识水平的学习者或不同的人工智能工具版本之间没有显著差异。为了优化GAI在教育中的应用，该研究建议将GAI与特定学科需求相结合，为不同学生水平调整工具，将GAI与传统教学相结合，并建立监测机制。这些战略旨在最大限度地发挥GAI对整个教育环境的学习效率和质量的积极影响。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope