Gesture recognition from depth images using motion and shape features

Abstract

In this paper, we propose an effective method to recognize 3D gestures from depth images which provide additional body motion and shape features. We project depth images onto three orthogonal planes and calculate the Motion History Image (MHI) of each projection to generate the 3 views MHI (3VMHI). Pyramid Histogram of Oriented Gradients (PHOG) is used to extract the features of the 3VMHI. Then, 3VMHI and PHOG are used together as a combined spacetime descriptor for gesture recognition. We provide a method to extract different gestures from a single video. Consecutive frame difference is employed to perform informative frame selection, which is able to remove uninformative frames. The experimental results on two challenging datasets demonstrate that our approach is effective and efficient.