Compound Facial Emotion Recognition based on Facial Action Coding System and SHAP Values

Abstract

Human facial emotion recognition is a difficult task in computer-human inter-action. Facial emotion recognition is required in many applications like med-ical, security, video games, e-physiotherapy, and counselling. Literature has many studies that have focused only on 6 basic emotions but advanced studies suggest human emotions are not limited to these 6 basic emotions. A human face can exhibit many other emotions, which are generated by combining the two basic emotions, these derived emotions are known as compound emotions. Recognition of compound emotions is also a very important task; hence this study proposes the use of the Facial Action Coding System (FACS) to identify 12 compound emotions. The authors identified and derived the intensities of 17 AUs with Openface library. Finally, two machine learning classifiers SVM (Support Vector Machine) and KNN (K-nearest neighbour) were implemented to identify 12 compound emotions, and results were compared. The experimental results show that the SVM classifier outperformed with an emotion recognition rate of 98.31% while the recognition rate of K-NN was