A novel biologically plausible spiking convolutional capsule network with optimized batch normalization for EEG-based emotion recognition

Abstract

Emotion recognition based on Electroencephalogram (EEG) is currently a hot topic in human-computer interaction, as EEG can more accurately reflect the characteristics of emotion. In recent years, EEG emotion recognition based on deep learning has achieved significant progress, and particularly methods combined with a capsule network (CapsNet) have outstanding performance. However, as the complexity of models continues to increase, resource consumption has also escalated. In this context, spiking neural networks (SNNs), known for their energy efficiency and greater biological plausibility, have attracted attentions of numerous researchers. Nevertheless, convolutional neural networks (CNNs) -based methods are not effective in SNN, and there remains a considerable gap compared with artificial neural networks (ANNs). This paper proposes a novel method combining the high-performing capsule networks with SNNs, named the spiking convolutional capsule network (SCCapsNet) for EEG-based emotion recognition tasks. To our knowledge, this is the first attempt to introduce capsule networks into SNNs for EEG emotion recognition. Furthermore, the spike-timing-dependent plasticity (STDP) routing algorithm is improved to sensitively capture temporal sequence information of EEG signals to enhance biological plausibility of SCCapsNet. In addition, a novel batch normalization (BN) layer incorporating the membrane potential decay time constant (tau-BN) is suggested to address the issue of neuron death caused by reduction in spike firing rate due to the $L_2$ norm. We provide a theoretical explanation of the role of the BN layer in regulating spike firing rates. Finally, the performance of SCCapsNet is validated on two public datasets. As for DEAP dataset, SCCapsNet achieved recognition accuracies of 97.01 %, 96.84 %, and 96.73 % for valence, arousal, and dominance dimensions, respectively. The accuracies are 89.82 %, 93.69 %, and 93.90 % on the same dimensions for DREAMER dataset. A recognition accuracy of 90.32 % was achieved on the five-category dataset SEED-V. Experimental results outperform all other comparable SNN methods. In addition, we validated the enhancing effect of the proposed tau-BN on spike firing rates. The results showed that the enhancement effect was obvious, successfully addressing the issue of neuron death caused by excessively low spike firing rates due to the $L_2$ norm. Our code is available at https://github.com/Zihao0/SCCapsNet.