GLADA: Global and Local Associative Domain Adaptation for EEG-Based Emotion Recognition

Abstract

Emotion recognition based on electroencephalography (EEG) has significant advantages in terms of reliability and accuracy. However, individual differences in EEG limit the ability of sentiment classifiers to generalize across subjects. Furthermore, due to the nonstationarity of EEG, subject signals can vary with time, an important challenge for temporal emotion recognition. Several emotion recognition methods have been developed that consider the alignment of conditional distributions, but do not balance the weights of conditional and marginal distributions. In this article, we propose a novel approach to generalize emotion recognition models across individuals and time, i.e., global and local associative domain adaptation (GLADA). The proposed method consists of three parts: 1) deep neural networks are used to extract deep features from emotional EEG data; 2) considering that marginal and conditional distributions between domains can contribute to adaptation differently, a method that combines coarse-grained adversarial adaptation and fine-grained adversarial adaptation is used to narrow the domain distance of the joint distribution in the EEG data between subjects (i.e., reduce intersubject variability), and the weights of the marginal and conditional distributions are automatically balanced using dynamic balancing factors; and 3) domain adaptation is used to accelerate model convergence. Using GLADA, subject-independent EEG emotion recognition is improved by reducing the influence of the subject’s personal information on EEG emotion. Experimental results demonstrate that the GLADA model effectively addresses the domain transfer problem, resulting in improved performance across multiple EEG emotion recognition tasks.