On-line human action recognition by combining joint tracking and key pose recognition

Abstract

In this paper, we present a boosting approach by combining the pose estimation and the upper body tracking to on-line recognize human actions. Instead of using a predefined pose to initialize the human skeleton, we construct a key poses database with depth HOG features as searching indexes. When user enters the camera view, we automatically search the database to get the initial skeleton. Then we use the particle filter to track human upper body parts. At the same time, we feed the tracking joints into the hidden Markov models to on-line spot and recognize the performed action. In order to rectify tracking errors, we apply the action recognition results and reuse our key poses database to reinforce the tracking process. Our contributions of the proposed approach are three-fold. First, our method can recognize human poses and actions at the same time. Second, with the key poses database and action recognition results as the feedback, the tracking process becomes more efficient and accurate. Third, we propose a spotting method based on the gradient of HMM probabilities, which thus enables our method to achieve on-line spotting and recognition. Experimental results demonstrate the effectiveness of the proposed approach.