Person-centered approach to explore learner's emotionality in learning within a 3D narrative game

Emotions form an integral part of our cognitive function. Past research has demonstrated conclusive associations between emotions and learning achievement [7, 26, 27]. This paper used a person-centered approach to explore students' (N = 65) facial behavior, emotions, learner traits and learning. An automatic facial expression recognition system was used to detect both middle school and university students' real-time facial movements while they learned scientific tasks in a 3D narrative video game. The results indicated a strong statistical relationship between three specific facial movements (i.e., outer brow raising, lip tightening and lip pressing), student self-regulatory learning strategy and learning performance. Outer brow raising (AU2) had strong predictive power when a student is confronted with obstacles and does not know how to proceed. Both lip tightening and pressing (AU23 and AU24) were predictive when a student engaged in a task that requires a deep level of incoming information processing and short memory activation. The findings also suggested a correlational relationship between student self-regulatory learning strategy use and neutral state. It is hoped that this study will provide empirical evidence for helping us develop a deeper understanding of the relationship between facial behavior and complex learning especially in educational games.