A Dynamic Gesture Recognition Method Based on Encoded Video

Abstract

Most of the video-based dynamic gesture recognition methods require decoding video into raw RGB images. The approved accuracy relies on multiple data patterns, such as depth map or optical flow, in specific scenario. So, the more complexity models, the huger calculation power and storage consumption. In this paper, a new characterized model for spatiotemporal data is proposed to represent the spatiotemporal features of dynamic gestures, take advantage of Intra-frames (I-frame), motion vectors, and residuals in encoded videos, so that the additional consumption of computation and storage caused by decoding videos are escaped. Furthermore, a key predicted frames (P-frame) selection (KPFS) module is proposed to filter those P-frames having no useful information, based on an image entropy estimated with the residuals. The more distinguished features are obtained. Comprehensively experiments are performed on two benchmark datasets, VIVA and SKIG. The results show that our method can achieve an average accuracy of 81.13% and 98.70% using lone RGB data, reduce the storage overhead by 88.5%. The result is similar to that of the state-of-the-art methods with the running speed of more than 4.3 times.