Facial expression and head gesture recognition using temporal self-similarity and bag of words of facial landmarks

Abstract

Automatic recognition of facial expressions and head gestures plays an important role in a wide range of research area including sign language recognition and human-computer interaction. In this work, we adopt the well-performing self-similarity based action recognition method to classify facial expressions and head gestures. Additionally, we propose a novel approach for facial gesture recognition based on the histogram of tracked facial landmarks. We fuse the presented techniques with our previous Hidden Markov Model based approach [1] and get 15% increase in classification results.