Effects of AI-generated adaptive feedback on statistical skills and interest in statistics: A field experiment in higher education

Abstract

This study explores whether AI-generated adaptive feedback or static feedback is favourable for student interest and performance outcomes in learning statistics in a digital learning environment. Previous studies have favoured adaptive feedback over static feedback for skill acquisition, however, without investigating the outcome of students' subject-specific interest. This study randomly assigned 90 educational sciences students to four conditions in a 2 × 2 Solomon four-group design, with one factor feedback type (adaptive vs. static) and, controlling for pretest sensitisation, another factor pretest participation (yes vs. no). Using a large language model, the adaptive feedback provided feedback messages tailored to students' responses for several tasks on reporting statistical results according to APA style, while static feedback offered a standardised expert solution. There was no evidence of pretest sensitisation and no significant effect of the feedback type on task performance. However, a significant medium-sized effect of feedback type on interest was found, with lower interest observed in the adaptive condition than in the static condition. In highly structured learning tasks, AI-generated adaptive feedback, compared with static feedback, may be non-essential for learners' performance enhancement and less favourable for learners' interest, potentially due to its impact on learners' perceived autonomy and competence.