An efficient multi modal emotion recognition system: ISAMC

Abstract

This paper presents a fusion approach called Image and Signal Analysis of Multimedia Content (ISAMC) to provide a fully evolved model for emotion recognition using both external (face) and internal (EEG signals) characteristics for the same emotional phenomenon. Both image analysis and EEG signal analysis is done using a video stimulus and based on wavelet approach for feature extraction. This novel methodology provides cross-validation of EEG and Image results with self-assessment of the participants and encourages multi-classification with the use of two different classifiers. The encouraging experimental results prove that the efficiency of this method is very high and due to its simplicity it can be a promising tool for emotion recognition.