Sequential learning for multimodal 3D human activity recognition with Long-Short Term Memory

Abstract

Capability of recognizing human activities is essential to human robot interaction for an intelligent robot. Traditional methods generally rely on hand-crafted features, which is not strong and accurate enough. In this paper, we present a feature self-learning mechanism for human activity recognition by using three-layer Long Short Term Memory (LSTM) to model long-term contextual information of temporal skeleton sequences for human activities which are represented by the trajectories of skeleton joints. Moreover, we add dropout mechanism and L2 regularization to the output of the three-layer Long Short Term Memory (LSTM) to avoid overfitting, and obtain better representation for feature modeling. Experimental results on a publicly available UTD multimodal human activity dataset demonstrate the effectiveness of the proposed recognition method.