Multimodal emotion recognition based on kernel canonical correlation analysis

Abstract

In order to deal with the limitation of the unmoral biometric systems, a multimodality emotion recognition system is proposed based on kernel canonical correlation analysis (KCCA). Because audio signal and facial expressions are two main channels of emotional communication, this approach extracts prosodic features and the visual features in FrFT domain. Those features are fused for the emotion recognition. The experimental results show that the multimodal recognition outperforms the unmoral biometric recognition.