Investigating Electrodermography (EDG) and Heart Rate (HR) Signals for Emotion Classification in Virtual Reality

Abstract

This paper shows the findings and results of combining Heart Rate and Electrodemography signals with KNN classifier for multimodal emotion detection using Virtual Reality (VR). The study was conducted by using a VR headset to show the participants 360 videos to elicit their emotional responses to the video. Their emotional response was captured using a wearable device that records both heart rate and skin activity in real time. A total of 5 participants took part in the experiment where the results are classified for intra-subject classification using KNN classifier. In the study, for the classification of intra-subject, the peak accuracy achieved amongst the five participants is 97.7% with HR and EDG signals combined with KNN as the classifier. These results demonstrate that by fusing HR and EDG signals, high-accuracy results can be generated through the use of KNN as the classifier. The application and potential of this study can be useful in entertainment, VR rehabilitation, and gaming.