Emotion agent: Unsupervised deep reinforcement learning with distribution-prototype reward for continuous emotional EEG analysis

Abstract

Continuous Electroencephalography (EEG) signals are widely employed in affective brain-computer interface (aBCI) applications. However, only a subset of the continuously acquired EEG data is truly relevant to emotional processing, while the remainder is often noisy or unrelated. Manual annotation of these key emotional segments is impractical due to their dynamic and individualized nature. To address this challenge, we propose a novel unsupervised deep reinforcement learning framework, termed Emotion Agent, which automatically identifies and extracts the most informative emotional segments from continuous EEG signals. Emotion Agent initially utilizes a heuristic algorithm to perform a global search and generate prototype representations of the EEG signals. These prototypes guide the exploration of the signal space and highlight regions of interest. Furthermore, we design a distribution-prototype-based reward function that evaluates the interaction between samples and prototypes to ensure that the selected segments are both representative and relevant to the underlying emotional states. Finally, the framework is trained using Proximal Policy Optimization (PPO) to achieve stable and efficient convergence. Experimental results on three widely used datasets (covering both discrete and dimensional emotion recognition) show an average improvement of 13.46 % when using the proposed Emotion Agent, demonstrating its significant enhancement of accuracy and robustness in downstream aBCI tasks.