Machine learning for hand gesture recognition using bag-of-words

Abstract

Human Computer Interaction received a great deal of attention this last decade. Last researches has turned to more natural interaction systems like gestural human machine interfaces. Recent works are attempting to solve the problem of hand gestures recognition using machine learning methods. Some of them are pretending to achieve very high performance. However, few of them are taking into account mandatory requirements to apply the workflow of a learning model, mainly data unbalance, model selection and generalization performance metric choice. In this work, we proposed a machine learning method for real time recognition of 16 gestures of user hands using the Kinect sensor that respects such requirements. The recognition is triggered only when there is a moving hand gesture. The method is based on the training of a Support Vector Machine model on hand depth data from which bag of words of SIFT and SURF descriptors are extracted. The data was kept balanced and the model kernel and parameters were selected using cross validation procedure. The method achieved 98% overall performance using the area under the ROC curve measure.