Learning to identify facial expression during detection using Markov decision process

Abstract

While there has been a great deal of research in face detection and recognition, there has been very limited work on identifying the expression on a face. Many current face detection methods use a Viola-Jones style "cascade" of Adaboost-based classifiers to detect faces. We demonstrate that faces with similar expression form "clusters" in a "classifier space" defined by the real-valued outcomes of these classifiers on the images and address the task of using these classifiers to classify a new image into the appropriate cluster (expression). We formulate this as a Markov decision process and use dynamic programming to find an optimal policy - here a decision tree whose internal nodes each correspond to some classifier, whose arcs correspond to ranges of classifier values, and whose leaf nodes each correspond to a specific facial expression, augmented with a sequence of additional classifiers. We present empirical results that demonstrate that our system accurately determines the expression on a face during detection