Research on Emotion Recognition Based on Facial Expression and EEG

With the development of artificial intelligence technology, emotion recognition has become an increasingly important research topic. Recognizing emotions only from the data with a single modality has its drawbacks. In this paper, the two modalities of facial expressions and EEG are integrated to realize the recognition of five types of emotions such as happiness, and the accuracy rate has reached a relatively satisfactory result. For facial expression modalities, this paper uses histogram equalization for preprocessing, then use LBP algorithm to extract facial expression features, and finally use SVM for expression recognition; for EEG modalities, this paper uses wavelet threshold denoising for preprocessing, and then use fractal dimension and multi-scale entropy algorithm to extract EEG signal features. This paper classifies EEG signals in the DEAP data set for emotion classification. Under the condition of using only one EEG channel FP1, the accuracy of SVM classification can reach 75.0%.