Assisting the autistic with improved facial expression recognition from mixed expressions

Abstract

People suffering from autism have difficulty with recognizing other people's emotions and are therefore unable to react to it. Although there have been attempts aimed at developing a system for analyzing facial expressions for persons suffering from autism, very little has been explored for capturing one or more expressions from mixed expressions which are a mixture of two closely related expressions. This is essential for psychotherapeutic tool for analysis during counseling. This paper presents the idea of improving the recognition accuracy of one or more of the six prototypic expressions namely happiness, surprise, fear, disgust, sadness and anger from the mixture of two facial expressions. For this purpose a motion gradient based optical flow for muscle movement is computed between frames of a given video sequence. The computed optical flow is further used to generate feature vector as the signature of six basic prototypic expressions. Decision Tree generated rule base is used for clustering the feature vectors obtained in the video sequence and the result of clustering is used for recognition of expressions. The relative intensity of expressions for a given face present in a frame is measured. With the introduction of Component Based Analysis which is basically computing the feature vectors on the proposed regions of interest on a face, considerable improvement has been noticed regarding recognition of one or more expressions. The results have been validated against human judgement.