EEG-CNN-Souping: Interpretable emotion recognition from EEG signals using EEG-CNN-souping model and explainable AI

Abstract

Emotion recognition is a key aspect of human–robot interaction (HRI), which requires social intelligence to perceive and react to human affective states. This paper introduces EEG-CNN-Souping, a novel approach that applies the “Model Soups” technique to a self-designed EEG-CNN model for classifying electroencephalogram (EEG) signals into emotions. EEG-CNN-Souping improves the model performance and efficiency by averaging the weights of multiple EEG-CNN models trained on different sizes of scalograms, which are acquired by applying continuous wavelet transform (CWT) and normalization to the EEG signals. The scalograms capture the time-varying patterns of the EEG signals effectively. The approach also uses data augmentation and gradient class activation map (Grad-Cam) visualization for robustness and interpretability respectively. The model is evaluated on a common dataset that is the SEED dataset and achieves a 99.31% accuracy, surpassing other state-of-the-art deep learning (DL) models in terms of accuracy, computational cost, and time efficiency. The prediction time for EEG-CNN-Souping is only 6 ms. The explainable artificial intelligence (XAI) method Grad-CAM is utilized for interpretation of predictions. EEG-CNN-Souping is computationally inexpensive and time-efficient.