Simultaneous Tracking and Recognition of Dynamic Digit Gestures for Smart TV Systems

Abstract

One of the most important digital home technologies is the television and its great development trends, which have been covering more and more exciting applications for family life. In this paper, we present a novel framework for simultaneous tracking and recognition of dynamic digit gestures, which can be further developed as part of a vision-based interface for smart television systems. A video stream capturing the gesture of a user will be fed into the proposed framework and the digit number inferred from the gesture will be output as a result. The trajectory of gesture is found using a fast and robust tracker, which is tuned by prior constraints and assisted by an motion predictor. The recognizer that connects with the tracker and turns the trajectory into the final output, is formed by the combination of dynamic time warping and the convolutional neural net. Experiments are conducted to validate and demonstrate the effectiveness of our proposed framework.