Semi-supervised pairwise transfer learning based on multi-source domain adaptation: A case study on EEG-based emotion recognition

Abstract

Negative transfer mitigation in transfer learning and universal-model establishment are crucial in electroencephalography (EEG)-based emotion recognition research. This study proposed a multi-source domain adaptation pairwise transfer learning method (named PLMSDANet) for EEG-based emotion recognition. PLMSDANet reduced the impact of negative transfer using a semi-supervised strategy that introduced a limited set of target-labeled data and selected the most compatible source domains for further training. In addition, a two-stage feature extractor was employed. Initially, we used a general feature extractor to capture the common spatial–spectral features of all the domains. Subsequently, we created independent branches for each pair of source and target domains to learn specific features from each source domain, incorporating discrepancy loss to harmonize the classification results of the different source domains. Furthermore, pairwise learning was used to solve the problem of the intra-domain distribution of sample classes. Finally, a cross-subject strategy was used to validate the public datasets SEED and SEED-IV extensively, achieving average emotion recognition accuracies of 90.09% and 73.08%, respectively. In summary, PLMSDANet combines multi-source domain transfer learning with semi-supervised paired learning methods, effectively transferring knowledge from multiple source domains to the target domain while enhancing the distinguishability between classes. The experimental results show that the PLMSDANet method effectively mitigates the negative-transfer issue and demonstrates excellent recognition performance, surpassing that of state-of-the-art methods.