A dual-branch self-supervised contrastive learning framework for emotion recognition based on time-frequency fusion

Abstract

Emotion recognition based on electroencephalography(EEG) signals is becoming a prominent research hotspot due to its wide-ranging applications in brain–computer interfaces (BCIs), mental health assessment, and human-computer interaction. Traditional emotion recognition methods often rely on supervised learning, which requires large amounts of labeled data to effectively train deep models. However, EEG signals exhibit inherent complexity and substantial variability across individuals and sessions, making it challenging to obtain consistent and reliable labels. In this paper, we propose a novel pretraining framework for EEG-based emotion recognition that enables mutual learning between time-domain and time-frequency-domain representations, while requiring simple network architectures. Experimental results demonstrate that our method achieves 84.39 % accuracy on the SEED dataset, and 89.01 % valence accuracy and 79.75 % arousal accuracy on the DEAP dataset using only 10 % labeled data, indicating strong performance under limited label conditions. Furthermore, we evaluate the transfer learning capability of our framework by pretraining it on the SEED dataset and then fine-tuning it on SEED-V. This cross-dataset transfer leads to a 1.9 % absolute improvement in classification accuracy on SEED-V, demonstrating the effectiveness of the learned representations in generalizing across datasets.