Multi-modal Emotion Recognition for Determining Employee Satisfaction

Abstract

Emotion recognition has been popular in the field of research for quite a while now. In this paper bi-modal emotion detection has been used to find employee satisfaction in workplaces. Interviews were taken of employees from different workplaces and were recorded. The recorded interviews were then used to detect the emotions of the employees from which their satisfaction level was derived. From the interviews, six different entities of emotion were detected, which are: Happiness, Sadness, Neutral, Disgust, Anger and Surprise. Two separate independent neural networks have been utilised. In one the facial expressions were detected and in the other sentiment analysis was done on the speech of the interviews after converting it into text. The extracted emotions were then fed into a Support Vector Machine (SVM) for determining the satisfaction of the employees. The satisfactions were categorized into five different levels which are: Highly dissatisfied, Dissatisfied, Neutral, Satisfied and Highly satisfied.