Multi-Stream CNN-LSTM Network with Partition Strategy for Human Action Recognition

Abstract

The wide application of human action recognition in the field of computer vision makes it a hot research topic in the past decades. In recent years, the prevalence of deep sensors and the proposal of real-time skeleton estimation algorithm based on deep images make human action recognition based on skeleton sequence attract increasing attention of researchers. Most of the existing work is aimed at extracting the spatial information of different joint nodes in a frame, but they do not fully consider the combination of temporal and spatial features. At the same time, the different joints were regarded as equally significant in most previous work, which is obviously not in line with the physiological characteristics and kinematics of human body. Therefore, in this paper, a human joint partition strategy is proposed to divide 25 human joints. In addition, a cnn-lstm framework is designed, which can simultaneously model the spatio-temporal characteristics of human skeleton sequence data, and extract the spatial domain information of different joints in a frame and the temporal domain information embedded in consecutive frames.