Cross-dataset EEG emotion recognition based on pre-trained Vision Transformer considering emotional sensitivity diversity

Abstract

As a crucial task in brain–computer interfaces, emotion recognition helps develop a profound understanding of human behaviour and mental health. Despite the development of various methods for EEG (electroencephalograph) emotion recognition, designing a model for effective cross-dataset emotion recognition remains challenging. To tackle this challenge, a transfer learning framework is introduced, which is referred to as Pre-trained Encoder from Sensitive Data (PESD). It involves a pre-training model on subjects with the highest emotional sensitivity. The trained model is then transferred to other datasets and subjects through a combination of three data alignment strategies: Mixup, Triplet loss, and Domain discriminator. The model is evaluated on four public datasets (SEED, SEED-IV, DEAP, and FACED) to achieve cross-dataset emotion recognition across all these four datasets. The highest accuracy results are 93.14% (SEED), 83.18% (SEED-IV), 93.53% (DEAP), and 92.55% (FACED), respectively. These results demonstrate significant improvement of our approach over existing ones in cross-dataset emotion recognition. The source code of this work is publicly available at https://github.com/fangwangeeg/PESD.