Predicting occluded skeletal joints via tracking-based feature extraction

Abstract

A rapidly growing research area in computer vision is the recognition of human poses, which imposes strong occlusion problems in subsequent tracking. Therefore, tracking mechanisms should contain algorithms for handling occlusions such that the skeletal data is continuous between frames. This paper introduces a new skeletal tracking approach where a deep-learning-based model for a Skeleton Feature Extractor is embedded into the tracking algorithm. This differs from the common tracking methods that, in the process of image feature extraction, consider feature extraction at joints and the spatial relation between them, thus making occlusion scenarios highly detectable. We further make a comparison with our model and MotioNet, which is a broadly applied model for 3D motion reconstruction. MotioNet can interpolate the missing joints based on the information both spatial and temporal. It, however, does not work when actual joints are occluded for some frames. Our model predicts the skeletal joints that are missing. Experiments on the JHMDB and Penn_Action dataset were meant to show that the method improves the accuracy of forecasting occluded skeletal joint positions by the same PCK metric.