A Novel Deep Learning Based Emotion Recognition Approach to well Being from Fingertip Blood Volume Pulse

Abstract

Emotions are central to physical and mental health and general well being. There is a great need to affordably and non invasively track moment to moment changes in emotional states and their conversion into chronic conditions. Blood Volume Pulse (BVP) is a widely used sensor for measuring blood volume changes, heart rate, and is embedded in numerous biofeedback systems and applications. Nonetheless, the role of BVP features relating to emotion detection is lacking in current studies. While engineers have become more interested in the analysis of heart rate variability (HRV) and its regulation by the autonomic nervous system, there is a need to design systems that can investigate their variations due to real life stressors and how people respond to emotions differently. The study employs the database for emotion analysis using physiological signals (DEAP) in assessing emotional responses of subjects according to valence arousal scale to music videos. We demonstrate a novel approach to augmenting original features and normalized features of blood volume in peripheral vessels. The features of HRV include tachogram, multi-scale entropy (MSE), power spectral density (PSD), and statistical moments derived from BVP. We further propose embedding age and gender of participants as a weight to the augmented features. We finally used multilayer perceptron (MLP) as classifier to evaluate our approach. Obtained results show an 8.4% and 7.3% improvement in F1-score in the valence and arousal dimension respectively. Such advances may aid in building closed-loop emotion detection and intervention systems.