Temporal-spectral-spatial synchronization attention-based network for EEG emotion recognition

Abstract

Electroencephalogram (EEG) provides an objective and precise representation of human emotional states, establishing EEG-based emotion recognition as a pivotal area in affective computing and intelligent systems. Nevertheless, EEG signals contain temporal-spectral-spatial features, exhibiting dynamic variations, frequency-band correlations, and spatial dependencies, with varying resolutions across domains. The challenge lies in adapting to resolution differences between domains, thereby improving the model’s ability to integrate complementary information across these domains. Moreover, processing multi-domain features often leads to complex model structures and excessive feature fusion, resulting in information loss. To tackle these challenges, we propose a unified framework: the Temporal-Spectral-Spatial Synchronization Attention-Based Network, which facilitates efficient modeling of multi-domain data. Specifically, the proposed network consists of a temporal-spectral-spatial attention encoder and a categorical decoder. The encoder adapts to resolution differences across temporal-spectral-spatial domains and synchronizes the fusion of spatiotemporal and spectral data, thus simplifying the model structure. Furthermore, we introduce a gating mechanism to adaptively balance the weights across domains and prevent excessive fusion that results in information loss. Finally, extensive experimental comparisons along with both subjective and objective analyses, demonstrate that our proposed network outperforms state-of-the-art models on the SEED, SEED-IV and DEAP.