Static and Dynamic Hand Gesture Recognition Using CNN Models

Abstract

Similar to the touchscreen, hand gesture based human computer interaction (HCI) is a technology that could allow people to perform a variety of tasks faster and more conveniently. This paper proposes a training method of an imaged-based hand gesture image and video clip recognition system using Convolutional Neural Networks (CNN). A dataset containing images of 6 different static hand gestures is used to train a 2D CNN model. ~98% accuracy is achieved. Furthermore, a 3D CNN model is trained on a dataset containing video clips of 4 dynamic hand gestures resulting in ~83% accuracy. This research demonstrates that a Cozmo robot loaded with pre-trained models is able to recognize static and dynamic hand gestures.