Biosignal Contrastive Representation Learning for Emotion Recognition of Game Users

Abstract

Biosignal representation learning (BRL) plays a crucial role in emotion recognition for game users (ERGU). Unsupervised BRL has garnered attention considering the difficulty in obtaining ground truth emotion labels from game users. However, unsupervised BRL in ERGU faces challenges, including overfitting caused by limited data and performance degradation due to unbalanced sample distributions. Faced with the above challenges, we propose a novel method of biosignal contrastive representation learning (BCRL) for ERGU, which not only serves as a unified representation learning approach applicable to various modalities of biosignals but also derives generalized biosignals representations suitable for different downstream tasks. Specifically, we solve the overfitting by introducing perturbations at the embedding layer based on the projected gradient descent (PGD) adversarial attacks and develop the sample balancing strategy (SBS) to mitigate the negative impact of the unbalanced sample on the performance. Further, we have conducted comprehensive validation experiments on the public dataset, yielding the following key observations: first BCRL outperforms all other methods, achieving average accuracies of 76.67%, 71.83%, and 63.58% in 1D-2 C Valence, 1D-2 C Arousal, and 2D-4 C Valence/Arousal, respectively; second, the ablation study shows that both the PGD module (+7.58% in accuracy on average) and the SBS module (+14.60% in accuracy on average) have a positive effect on the performance of different classifications; third, BCRL model exhibits the certain generalization ability across the different games, subjects and classifiers.