A temporal-spatial feature fusion network for emotion recognition with individual differences reduction.

Abstract

Purpose: In the context of EEG-based emotion recognition tasks, a conventional strategy involves the extraction of spatial and temporal features, subsequently fused for emotion prediction. However, due to the pronounced individual variability in EEG and the constrained performance of conventional time-series models, cross-subject experiments often yield suboptimal results. To address this limitation, we propose a novel network named Time-Space Emotion Network (TSEN), which capitalizes on the fusion of spatiotemporal information for emotion recognition.

Methods: Diverging from prior models that integrate temporal and spatial features, our network introduces a Convolutional Block Attention Module (CBAM) during spatial feature extraction to judiciously allocate weights to feature channels and spatial positions. Furthermore, we bolster network stability and improve domain adaptation through the incorporation of a residual block featuring Switchable Whitening (SW). Temporal feature extraction is accomplished using a Temporal Convolutional Network (TCN), ensuring elevated prediction accuracy while maintaining a lightweight network structure.

Results: We conduct experiments on the preprocessed DEAP dataset. Ultimately, the average accuracy for arousal prediction is 0.7032 with a variance of 0.0876, and the F1 score is 0.6843. For valence prediction, the accuracy is 0.6792 with a variance of 0.0853, and the F1 score is 0.6826.

Conclusion: TSEN exhibits high accuracy and low variance in cross-subject emotion prediction tasks, effectively reducing individual differences among different subjects. Additionally, TSEN has a smaller parameter count, enabling faster execution.