Examining learners’ cognitive processes and learning emotions in digital game-based vocabulary learning: evidence from eye-tracking technology and facial emotion recognition

Purpose This research explored the intersection of cognitive processes, emotions and their impacts on digital game-based vocabulary learning (DGVL) among university students. Recognizing the scant research in this area, especially with integrating innovative technologies, this study aims to understand the influence of these elements using advanced monitoring tools. Design/methodology/approach This inquiry was carried out as an observational study involving 44 university students segmented into three English language proficiency levels: high, intermediate and low based on their English course scores. The methodological tools included a portable eye tracker to observe visual behaviors and deep learning technology to identify and analyze the participants’ emotional responses and engagement with the DGVL during the learning process. Findings The results showed that distinct fixation sequences and variations in visual attention during DGVL were correlated with different levels of competency, suggesting a direct correlation between visual engagement and language competence. In addition, emotional transitions, predominantly from engagement (“flow”) to challenge (“frustration”), were common among participants, reflecting the emotional dynamics of learning. Furthermore, all participants consistently focused on the English vocabulary definitions, indicative of their targeted approach to understanding and test preparation. These findings highlighted the intricate dynamics between emotions and cognitive processes in learning environments. Originality/value Contribution of this study shows the interplay of cognitive engagement and emotional experiences in the context of DGVL. It underscored the complex nature of these factors and their collective influence on learners’ visual and emotional engagement, offering valuable implications for educational strategies and technological applications in language learning.