AI-Based Adaptive Feedback in Simulations for Teacher Education: An Experimental Replication in the Field

IF 5.1 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2025-01-15 DOI:10.1111/jcal.13123

Elisabeth Bauer, Michael Sailer, Frank Niklas, Samuel Greiff, Sven Sarbu-Rothsching, Jan M. Zottmann, Jan Kiesewetter, Matthias Stadler, Martin R. Fischer, Tina Seidel, Detlef Urhahne, Maximilian Sailer, Frank Fischer

{"title":"AI-Based Adaptive Feedback in Simulations for Teacher Education: An Experimental Replication in the Field","authors":"Elisabeth Bauer, Michael Sailer, Frank Niklas, Samuel Greiff, Sven Sarbu-Rothsching, Jan M. Zottmann, Jan Kiesewetter, Matthias Stadler, Martin R. Fischer, Tina Seidel, Detlef Urhahne, Maximilian Sailer, Frank Fischer","doi":"10.1111/jcal.13123","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Artificial intelligence, particularly natural language processing (NLP), enables automating the formative assessment of written task solutions to provide adaptive feedback automatically. A laboratory study found that, compared with static feedback (an expert solution), adaptive feedback automated through artificial neural networks enhanced preservice teachers' diagnostic reasoning in a digital case-based simulation. However, the effectiveness of the simulation with the different feedback types and the generalizability to field settings remained unclear.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>We tested the generalizability of the previous findings and the effectiveness of a single simulation session with either feedback type in an experimental field study.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>In regular online courses, 332 preservice teachers at five German universities participated in one of three randomly assigned groups: (1) a simulation group with NLP-based adaptive feedback, (2) a simulation group with static feedback and (3) a no-simulation control group. We analysed the effect of the simulation with the two feedback types on participants' judgement accuracy and justification quality.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusions</h3>\n \n <p>Compared with static feedback, adaptive feedback significantly enhanced justification quality but not judgement accuracy. Only the simulation with adaptive feedback significantly benefited learners' justification quality over the no-simulation control group, while no significant differences in judgement accuracy were found.</p>\n \n <p>Our field experiment replicated the findings of the laboratory study. Only a simulation session with adaptive feedback, unlike static feedback, seems to enhance learners' justification quality but not judgement accuracy. Under field conditions, learners require adaptive support in simulations and can benefit from NLP-based adaptive feedback using artificial neural networks.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 1","pages":""},"PeriodicalIF":5.1000,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jcal.13123","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.13123","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Artificial intelligence, particularly natural language processing (NLP), enables automating the formative assessment of written task solutions to provide adaptive feedback automatically. A laboratory study found that, compared with static feedback (an expert solution), adaptive feedback automated through artificial neural networks enhanced preservice teachers' diagnostic reasoning in a digital case-based simulation. However, the effectiveness of the simulation with the different feedback types and the generalizability to field settings remained unclear.

Objectives

We tested the generalizability of the previous findings and the effectiveness of a single simulation session with either feedback type in an experimental field study.

Methods

In regular online courses, 332 preservice teachers at five German universities participated in one of three randomly assigned groups: (1) a simulation group with NLP-based adaptive feedback, (2) a simulation group with static feedback and (3) a no-simulation control group. We analysed the effect of the simulation with the two feedback types on participants' judgement accuracy and justification quality.

Results and Conclusions

Compared with static feedback, adaptive feedback significantly enhanced justification quality but not judgement accuracy. Only the simulation with adaptive feedback significantly benefited learners' justification quality over the no-simulation control group, while no significant differences in judgement accuracy were found.

Our field experiment replicated the findings of the laboratory study. Only a simulation session with adaptive feedback, unlike static feedback, seems to enhance learners' justification quality but not judgement accuracy. Under field conditions, learners require adaptive support in simulations and can benefit from NLP-based adaptive feedback using artificial neural networks.

Abstract Image

查看原文本刊更多论文

基于人工智能的自适应反馈在教师教育模拟中的应用：一个实地的实验复制

人工智能，特别是自然语言处理（NLP），使书面任务解决方案的形成性评估自动化，从而自动提供自适应反馈。一项实验室研究发现，与静态反馈（专家解决方案）相比，通过人工神经网络自动化的自适应反馈在基于数字案例的模拟中增强了职前教师的诊断推理能力。然而，不同反馈类型的模拟的有效性和对现场设置的泛化性仍然不清楚。目的：在实验现场研究中，我们测试了先前研究结果的普遍性和单次模拟会话的有效性。方法在常规在线课程中，德国五所大学的332名职前教师随机分为三组：(1)基于nlp的自适应反馈模拟组，(2)静态反馈模拟组，(3)无模拟对照组。我们分析了两种反馈类型的模拟对被试判断准确性和证明质量的影响。结果与结论与静态反馈相比，自适应反馈显著提高了判断质量，但没有提高判断精度。只有具有自适应反馈的模拟显著地提高了学习者的判断质量，而在判断准确性方面没有发现显著差异。我们的实地实验重复了实验室研究的结果。与静态反馈不同，只有具有自适应反馈的模拟会话似乎能提高学习者的判断质量，但不能提高判断准确性。在现场条件下，学习器在模拟中需要自适应支持，并且可以使用人工神经网络从基于nlp的自适应反馈中获益。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope