Beyond Big Data of Human Behaviors: Modeling Human Behaviors and Deep Emotions

Abstract

Humans possess a variety of long term or short term behaviors such as gesture, posture, and movement and so on. These readable behaviors usually convey significant emotional information, which can facilitate human-machine interactions in intelligent cognitive systems. However, there is a lack of studies on modeling such complex relationship between human behavior and emotion in a time series context. This paper attempts to pioneer such an exploration. First, huge amounts of human behaviors are suggested to be captured by various sensors. Then behaviors and emotions are modeled by deep structure of bidirectional LSTM, which can represent interactions and correlations. To avoid training difficulties, bidirectional LSTM are only located in the bottom layer, and the other layers are uni-bidirectional, while the adjacent layers use residual connections. This deep bidirectional LSTM has the advantage that it can be scaled up to larger varieties of human behaviors captured by multiple sensors. The experimental results show that our proposed deep structure for modeling human behaviors and emotions is able to achieve a high degree of accuracy than shallow representation or models.