Analysis of student e-learning engagement using learning affect: Hybrid of facial emotions and domain model

E-learning offers the flexibility of learning time and location for many students than traditional education. However, not all learning courses can be easily learned through online platforms and effectively meet the students' needs. As a result, lead to a loss of learning motivation and concentration on the student's side. While most recent studies have considered facial emotions as an effective tool for interpreting learning experiences in learners, but they have ignored the characteristic of courses. And learner engagement and performance on specific knowledge units is an effective method to analyze the student learning process. Therefore, this study proposed a hybrid approach for analyzing student engagement in video-based online courses, including a knowledge map to represent knowledge units and their relationships, and a convolutional neural network to examine the learners' facial expressions to interpret their learning effect on each concept during e-learning. The classification process has been adapted to identify different grades of learning emotions at knowledge units. The knowledge map partition method has been proposed, and by analyzing and visualizing the different divisions, the teacher can better understand the student's understanding levels based on his emotion within the course. The study demonstrated how personalized reports of knowledge unit understanding from this model could serve as a basis for future course content modifications and the organization and optimizing teaching materials.