Emotion Recognition by Point Process Characterization of Heartbeat Dynamics

Abstract

Recognizing human emotion from heartbeat information alone is a challenging but ongoing research area. Here, we utilize a point process model to characterize heartbeat dynamics and use it to extract instantaneous heart rate variability (HRV) features. These features are then fed into a convolutional neural network (CNN) to characterize different emotional states from small windows. On average, we achieved over 60% classification accuracy and as high as 77% in some subjects. This is comparable to other studies that use a combination of physiological signals as opposed to only HRV measures as done here. Informative features were identified for the different affective states. These findings enable the possibility of augmenting electrocardiogram or photoplethysmogram monitoring wearable devices with automated human emotion recognition capabilities for mental health applications. They also allow for the use of instantaneous estimation of HRV features to be used in combination with models that use other types of physiological signals for instantaneous emotion recognition.