Skeleton-based Human Activity Analysis Using Deep Neural Networks with Adaptive Representation Transformation

Compared with RGB-D-based human action analysis, skeleton-based works reach higher robustness and better performance, which are widely applied in the real world. However, the diversity of action observation perspectives hinders the improvement of recognition accuracy. Most of the existing works solve this problem by increasing the amount of training data, which brings a huge computational cost and cannot improve the robustness of the models. This paper proposes an adaptive model to obtain high-performance representations to improve human action recognition accuracy. First, a skeleton representation transfer scheme is proposed to transform the input skeleton-based body model to the best perspective, in which all parameters can be adaptively learned. This is more robust and cost-effective than hand-crafted features. Next, a re-designed backbone is proposed to train the model with a small computational cost based on the 3D-CNN. In the training process, a data enhancement method is also introduced to enhance robustness. Finally, extensive experimental evaluations are conducted on two benchmarks. The results show that this deep model can effectively and adaptively obtain high-performance skeleton representation and its performance is better than other state-of-the-art methods.