Facial event mining using coupled hidden Markov models

Facial event mining is one of the key techniques for automatic human face analysis. It plays an important role in human computer interaction. This paper proposes a new approach to facial event recognition by combining active shape models (ASMs) and coupled hidden Markov models (CHMMs). Based on the assumption that a complex facial event can be decomposed into multiple coupled processes, ASMs are used to track global facial features and to decouple pattern attributes for upper and lower faces separately. These two interacting processes are modeled as a CHMM for training and recognition. Four basic facial events are investigated. Preliminary experiments yield consistent results that show the significant advantage of CHMMs over conventional HMMs for facial event mining in video.