Set-pMAE: spatial-spEctral-temporal based parallel masked autoEncoder for EEG emotion recognition

Abstract

The utilization of Electroencephalography (EEG) for emotion recognition has emerged as the primary tool in the field of affective computing. Traditional supervised learning methods are typically constrained by the availability of labeled data, which can result in weak generalizability of learned features. Additionally, EEG signals are highly correlated with human emotional states across temporal, spatial, and spectral dimensions. In this paper, we propose a Spatial-spEctral-Temporal based parallel Masked Autoencoder (SET-pMAE) model for EEG emotion recognition. SET-pMAE learns generic representations of spatial-temporal features and spatial-spectral features through a dual-branch self-supervised task. The reconstruction task of the spatial-temporal branch aims to capture the spatial-temporal contextual dependencies of EEG signals, while the reconstruction task of the spatial-spectral branch focuses on capturing the intrinsic spatial associations of the spectral domain across different brain regions. By learning from both tasks simultaneously, SET-pMAE can capture the generalized representations of features from the both tasks, thereby reducing the risk of overfitting. In order to verify the effectiveness of the proposed model, a series of experiments are conducted on the DEAP and DREAMER datasets. Results from experiments reveal that by employing self-supervised learning, the proposed model effectively captures more discriminative and generalized features, thereby attaining excellent performance.