The potential and limitations of large language models for automatic classification of teachers' motivational messages in educational research.

Abstract

Introduction: The rapid advancement of artificial intelligence (AI) has created new opportunities in educational research, particularly in the efficient analysis of complex social interactions within classrooms. One promising area involves the classification of teachers' motivational messages. Traditionally, such assessments have relied on self-reports and observer evaluations, which require a lot of staff and time resources. Recently, large language models (LLMs) have been employed to classify teachers' motivational messages, offering novel, less labour-intensive approaches for classification.

Aims: Building on these recent developments, this work presents a comprehensive literature overview exploring the applications, potential, and limitations of using LLMs to classify teachers' motivational messages.

Results: The present comprehensive literature overview indicates that the use of LLMs for classifying teachers' motivational messages is a promising yet still emerging field of research. Recent studies have applied LLMs in innovative ways, drawing on established motivational theories and employing novel classification techniques, such as zero-shot and few-shot prompting or fine-tuning, to classify motivational messages. Open questions remain, particularly concerning the structure, quantity, and quality of annotated material.

Discussion: Whereas recent studies have demonstrated the potential of LLMs to offer scalable and time-efficient alternatives for classifying motivational messages in the classroom, several challenges persist. These include concerns related to the quality and quantity of training data, model generalisability, the ability to capture the complexity of classroom interactions, and biases involved in integrating LLMs as a classification method. This comprehensive literature overview provides practical recommendations for the responsible use of LLMs in educational research and school practice.