A Continuous Real-time Hand Gesture Recognition Method based on Skeleton

Abstract

While isolated hand gesture recognition methods aims to determine the type of gestures for a given sequence, continuous hand gesture recognition methods have to perform one more task: determining the starting point and ending point of the hand gesture. This task becomes challenging as the starting point and ending points of the gestures are not usually obvious even for human being. This paper presents a method for continuous hand gesture recognition based on skeleton information that consists of two phases: gesture detection and gesture recognition. In our method, to leverage the lightweight and the robustness of recognition models, TD-Net (Triple Feature Double Motion) model is employed in both gesture detection and recognition phases. Experimental results on IPN dataset have shown that the proposed method outperforms different state-of-the-art methods with 40.10% of Levenshtein accuracy and 0.1ms of inference time.