Facial Emotion Recognition Based on Facial Motion Stream Generated by Kinect

Abstract

Nowadays, the human facial emotion recognition has been used in wide range of applications that is directly involved in a human life. Due to the fragility of humans, the performance in these applications has to be improved. In this paper, we describe the novel approach to extract the facial feature from moving pictures. We introduce the facial movement stream, which is derived from the distance measurement between each pair of the coordinates located on human facial wireframe flowing through each frame of the movement. We have proposed the Facial Emotion Recognition Based on Facial Motion Stream generated by Kinect employing two kinds of facial features. The first one was just a simple distance value of each pair-wise coordinates packed into 153-dimensional feature vector per frame. The second one was derived from the first one based on Structured Streaming Skeleton approach and it became 765-dimensional feature vector per frame. We have presented the method to construct the dataset by ourselves since there was no dataset available for our approach. The facial movements of five people were collected in the experiment. The result shows that the average accuracy of SSS feature outperformed the simple distance feature using K-Nearest Neighbors by 10% and that using Support Vector Machine by 26%.