Real-time emotions recognition system

Abstract

This paper presents the experimental setup and methodology for a real-time emotions recognition system, based on the recent Intel RealSense 3D sensor, to identify six emotions: happiness, sadness, anger, surprise, fear, and neutral. The process includes the database construction, with 43 participants, based on facial features extraction and a multiclass Support Vector Machine classifier. The system was first tested offline using Linear kernel and Radial Basis Function (RBF) kernel. In the offline evaluation, the system performance was quantified in terms of confusion matrix, accuracy, sensitivity, specificity, Area Under the Curve, and Mathews Correlation Coefficient metrics. The RBF kernel achieved the best performance, with an average accuracy of 93.6%. Then, the real-time system was evaluated in a laboratorial setup, achieving an overall accuracy of 88%. The time required for the system to perform facial expression recognition efficiently is 1–3ms. The results, obtained by simulation and experimentally, point out that the present system can recognize facial expressions accurately.