Engagement Emotion Classification through Facial Landmark Using Convolutional Neural Network

Abstract

The concept of engagement is the way people are involved in an activity that relates to emotional feelings and attention. In an e-learning environment, engagement can be used as a benchmark for evaluating learning activities because emotional involvement refers to students' affective reactions such as interest, boredom, confusion, or frustration. A person's emotions can be recognized through facial expressions. However, facial expression data of images have high dimensions, resulting in large computational time in the model learning process. To reduce computation time and data dimensions, feature extraction methods such as facial landmarks can be used. Therefore, this study aims to build an emotional engagement recognition system through facial landmarks by implementing the Convolutional Neural Network (CNN) method. Five facial landmarks and Euclidean distance between points and center point from the facial image dataset were detected which is then used as CNN training data. Based on the results of implementation and analysis, an average accuracy of 97.51% was obtained from CNN using five k-fold cross-validations. These results are compared with Deep Neural Network which achieved an average accuracy of 97.14% and SVM which achieved an average accuracy of 89.84%. The accuracy results obtained indicate that CNN successfully recognizes engagement emotion better than the other method.