A Multivisual Cue-Based Approach for Intelligent Assessment of Student Engagement in the Classroom

Abstract

Accurately assessing student engagement in classroom environments is essential for understanding learning processes and providing evidence to support instructional improvement. However, existing studies are often limited by restricted engagement dimensions, insufficient model generalizability, and a lack of process-oriented analysis. To address these challenges, this study proposes a multivisual cue-based framework for intelligent classroom engagement assessment by integrating object detection and tracking, facial expression recognition, and learning behavior recognition. To support model development and evaluation, a classroom-oriented academic emotion dataset containing 26 842 annotated facial-expression instances and a learning behavior dataset comprising 152 823 labeled behavior samples were constructed. Experimental results show that the facial expression recognition and learning behavior recognition models achieved accuracies of 84.1% and 91.4%, respectively. Based on the recognized emotional and behavioral states, emotional engagement, behavioral engagement, and overall engagement were dynamically assessed at both the individual and classroom levels. A case study conducted on authentic classroom videos further demonstrated the capability of the proposed framework to characterize the temporal evolution of student engagement and reveal engagement patterns across different instructional phases. These findings not only offer educators fine-grained insights into classroom dynamics but also establish a scalable foundation for real-time learning analytics, contributing to the advancement of intelligent classroom monitoring and pedagogical decision-making.