CycleMVAE: Benchmarking End-to-End Cycle-Consistent Multi-Task Variational Autoencoder for EEG-Based Emotion Recognition

Abstract

Affective computing, particularly the identification of emotions from multichannel electroencephalography (EEG) signals, has gained importance. In this study, we propose a novel deep neural network model called Cycle Consistent Multi-Task Variational Autoencoder (CycleMVAE) to simultaneously investigate pairwise translation of emotional features, signal reconstruction, and emotion classification across two different EEG recording samples. CycleMVAE comprises two Variational Autoencoders (VAEs) and a supervised classifier. Each VAE consists of an encoder and a decoder. The encoder of the first VAE transfers emotional properties from EEG sample X to a compact latent space Z, while the decoder retrieves these features from Z to transfer them to EEG sample Y. Similarly, the second VAE uses the compact latent space Z’ to transfer emotion features from EEG sample Y to EEG sample X. This forms a cyclic translation of feature among EEG sample recordings. The model is trained using reconstructed loss, cycle consistency loss, and latent vector regularization loss, with a supervised classifier used to categorize emotions into arousal, valence, and dominance categories. The proposed approach improves EEG classification performance while reducing pre-processing complexity. The effectiveness of the proposed approach has been validated by experimental findings on the multimodal DREAMER emotional database.