FBSTCNet: A Spatio-Temporal Convolutional Network Integrating Power and Connectivity Features for EEG-Based Emotion Decoding

Abstract

Electroencephalography (EEG)-based emotion recognition plays a key role in the development of affective brain-computer interfaces (BCIs). However, emotions are complex and extracting salient EEG features underlying distinct emotional states is inherently limited by low signal-to-noise ratio (SNR) and low spatial resolution of practical EEG data, which is further compounded by the lack of effective spatio-temporal filter optimization approaches for generic EEG features. To address these challenges, this study proposes a set of neural networks termed the Filter-Bank Spatio-Temporal Convolutional Networks (FBSTCNets) for performing end-to-end multi-class emotion recognition via robust extraction of power and/or connectivity features from EEG. First, a filter bank is employed to construct a multiview spectral representation of EEG data. Next, a temporal convolutional layer, followed by a depth-wise spatial convolutional layer, performs spatio-temporal filtering, transforming EEG into latent signals with higher SNR. A feature extraction layer then extracts power and/or connectivity features from the latent signals. Finally, a fully connected layer with a cropped decoding strategy predicts the emotional state. Experimental results on two public emotion EEG datasets, SEED and SEED-IV, demonstrate that FBSTCNets outperform previous benchmark methods in decoding accuracy. Our approach provides a principled emotion decoding framework for designing high-performance spatio-temporal filtering networks tailored to specific EEG feature types.