Development of a generative AI-powered teachable agent for middle school mathematics learning: A design-based research study

IF 8.1 1区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

British Journal of Educational Technology Pub Date : 2025-04-17 DOI:10.1111/bjet.13586

Wanli Xing, Yukyeong Song, Chenglu Li, Zifeng Liu, Wangda Zhu, Hyunju Oh

{"title":"Development of a generative AI-powered teachable agent for middle school mathematics learning: A design-based research study","authors":"Wanli Xing, Yukyeong Song, Chenglu Li, Zifeng Liu, Wangda Zhu, Hyunju Oh","doi":"10.1111/bjet.13586","DOIUrl":null,"url":null,"abstract":"<div>\n \n <section>\n \n <p>This paper reports on a design-based research (DBR) study that aims to devise an artificial intelligence (AI)-powered teachable agent that supports secondary school students' learning-by-teaching practices of mathematics learning content. A long-standing pedagogical practice of learning-by-teaching is powered by a recent advancement of generative AI technologies, yielding our teachable agent called <i>ALTER-Math</i>. This study chronicles one usability testing and three cycles of iterative design and implementation process of <i>ALTER-Math</i>. The three empirical studies involved a total of 320 middle school students and six teachers in authentic classroom settings. The first study was exploratory, focusing on the qualitative feedback from the students and teachers through open-ended surveys, interviews and classroom observations. The second study yielded a medium-high (<i>M</i> = 3.26) quantitative survey result on students' perceived engagement and usability on top of the qualitative findings. Finally, the final study included pre- and post-knowledge tests in a quasi-experimental study design as well as student and teacher interviews. The final study revealed a bigger significant knowledge improvement in students who used <i>ALTER-Math</i> compared to the control group, suggesting a positive impact of AI-powered teachable agents on students' learning. The design implications learned from multiple iterations are discussed to inform the future design of AI-powered learning technologies.</p>\n </section>\n \n <section>\n \n <div>\n \n <div>\n \n <h3>Practitioner notes</h3>\n <p>What is already known about this topic\n\n </p><ul>\n \n <li>Learning-by-teaching is a long-standing effective pedagogical strategy to enhance students' domain knowledge and feelings of responsibility in learning.</li>\n \n <li>Various teachable agents have been developed and have demonstrated benefits in students' learning.</li>\n \n <li>Generative AI offers the potential to provide naturalistic, contextualised and adaptive conversations.</li>\n </ul>\n <p>What this paper adds\n\n </p><ul>\n \n <li>Develops a novel generative AI-powered teachable agent for middle school mathematics learning, called <i>ALTER-Math</i>.</li>\n \n <li>Reports the iterative design process involving empirical classroom implementations of <i>ALTER-Math</i>.</li>\n \n <li>Reveals a bigger significant improvement in the student's mathematical knowledge after using <i>ALTER-Math</i>, compared to the control group.</li>\n </ul>\n <p>Implications for practice and/or policy\n\n </p><ul>\n \n <li>Researchers can be inspired by this design example of a theoretically grounded generative AI learning technology.</li>\n \n <li>Educational technology designers could hear the real voices of students and teachers about the generative AI learning technologies.</li>\n \n <li>Researchers and educational technology designers could be directed by the design implications to the future design of AI-powered learning technologies and teachable agents.</li>\n </ul>\n </div>\n </div>\n </section>\n </div>","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":"56 5","pages":"2043-2077"},"PeriodicalIF":8.1000,"publicationDate":"2025-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Journal of Educational Technology","FirstCategoryId":"95","ListUrlMain":"https://bera-journals.onlinelibrary.wiley.com/doi/10.1111/bjet.13586","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

This paper reports on a design-based research (DBR) study that aims to devise an artificial intelligence (AI)-powered teachable agent that supports secondary school students' learning-by-teaching practices of mathematics learning content. A long-standing pedagogical practice of learning-by-teaching is powered by a recent advancement of generative AI technologies, yielding our teachable agent called ALTER-Math. This study chronicles one usability testing and three cycles of iterative design and implementation process of ALTER-Math. The three empirical studies involved a total of 320 middle school students and six teachers in authentic classroom settings. The first study was exploratory, focusing on the qualitative feedback from the students and teachers through open-ended surveys, interviews and classroom observations. The second study yielded a medium-high (M = 3.26) quantitative survey result on students' perceived engagement and usability on top of the qualitative findings. Finally, the final study included pre- and post-knowledge tests in a quasi-experimental study design as well as student and teacher interviews. The final study revealed a bigger significant knowledge improvement in students who used ALTER-Math compared to the control group, suggesting a positive impact of AI-powered teachable agents on students' learning. The design implications learned from multiple iterations are discussed to inform the future design of AI-powered learning technologies.

Practitioner notes

What is already known about this topic

Learning-by-teaching is a long-standing effective pedagogical strategy to enhance students' domain knowledge and feelings of responsibility in learning.
Various teachable agents have been developed and have demonstrated benefits in students' learning.
Generative AI offers the potential to provide naturalistic, contextualised and adaptive conversations.

What this paper adds

Develops a novel generative AI-powered teachable agent for middle school mathematics learning, called ALTER-Math.
Reports the iterative design process involving empirical classroom implementations of ALTER-Math.
Reveals a bigger significant improvement in the student's mathematical knowledge after using ALTER-Math, compared to the control group.

Implications for practice and/or policy

Researchers can be inspired by this design example of a theoretically grounded generative AI learning technology.
Educational technology designers could hear the real voices of students and teachers about the generative AI learning technologies.
Researchers and educational technology designers could be directed by the design implications to the future design of AI-powered learning technologies and teachable agents.

Abstract Image

查看原文本刊更多论文

面向中学数学学习的生成式人工智能可教代理的开发：基于设计的研究性研究

本文报道了一项基于设计的研究（DBR）研究，该研究旨在设计一个人工智能（AI）驱动的可教代理，以支持中学生在数学学习内容的教学实践中学习。一种长期存在的教学实践是由最近的生成式人工智能技术的进步所推动的，产生了我们称为ALTER-Math的可教代理。本研究记录了ALTER-Math的一个可用性测试和三个迭代设计和实现周期。三个实证研究涉及320名中学生和6名教师在真实的课堂环境中。第一项研究是探索性的，通过开放式调查、访谈和课堂观察，重点关注学生和教师的定性反馈。第二项研究在定性调查结果的基础上，对学生的感知参与和可用性进行了中高（M = 3.26）的定量调查。最后，最后的研究包括准实验研究设计的知识前和知识后测试以及学生和教师访谈。最后的研究显示，与对照组相比，使用ALTER-Math的学生的知识水平有了更大的显著提高，这表明人工智能教学代理对学生的学习产生了积极影响。讨论了从多次迭代中学习到的设计含义，以便为人工智能学习技术的未来设计提供信息。关于这个话题，我们已经知道，教中学习是一种长期有效的教学策略，可以增强学生的领域知识和学习责任感。各种可教代理已被开发出来，并已证明对学生的学习有好处。生成式AI提供了提供自然、情境化和适应性对话的潜力。为中学数学学习开发了一种新的生成式人工智能可教代理，称为ALTER-Math。报告涉及ALTER-Math的经验课堂实施的迭代设计过程。与对照组相比，使用ALTER-Math后，学生的数学知识有了更大的显著提高。对实践和/或政策的启示研究人员可以从这个基于理论的生成式人工智能学习技术的设计示例中得到启发。教育技术设计师可以听到学生和老师对生成式人工智能学习技术的真实声音。研究人员和教育技术设计师可以通过设计影响来指导人工智能学习技术和可教代理的未来设计。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

British Journal of Educational Technology EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

15.60

自引率

4.50%

发文量

111

期刊介绍： BJET is a primary source for academics and professionals in the fields of digital educational and training technology throughout the world. The Journal is published by Wiley on behalf of The British Educational Research Association (BERA). It publishes theoretical perspectives, methodological developments and high quality empirical research that demonstrate whether and how applications of instructional/educational technology systems, networks, tools and resources lead to improvements in formal and non-formal education at all levels, from early years through to higher, technical and vocational education, professional development and corporate training.