Spatio temporal 3D skeleton kinematic joint point classification model for human activity recognition

Human activity recognition in video data is challenging due to factors like cluttered backgrounds and complex movements. This work introduces the Stacked Ensemble 3D Skeletal Human Activity Recognition (SES-HAR) framework to tackle these issues. The framework utilizes MoveNet Lightning Pose Estimation to generate 2D skeletal kinematic joint points, which are then mapped to 3D using a Gaussian Radial Basis Function Kernel. SES-HAR employs a stacking ensemble approach with two layers: level-0 base learners and a level-1 meta-learner. Base learners include Convolutional Two-Part Long Short-Term Memory Network (Conv2P-LSTM), Spatial Bidirectional Gated Temporal Graph Convolutional Network (SBGTGCN) with attention, and Convolutional eXtreme Gradient Boosting (ConvXGB). Their outputs are pooled and processed by a Logistic Regression (LR) meta-learner in the level-1 layer to generate final predictions. Experimental results show that SES-HAR achieves significant performance improvements on NTU-RGB + D 60, NTU-RGB + D 120, Kinetics-700–2020, and Micro-Action-52 datasets.