Hand Gesture Localization and Classification by Deep Neural Network for Online Text Entry

Abstract

Hand gesture recognition is an important aspect of human-computer interaction. A proper hand gesture recognizing system can be used to build a robust text entry system for human-computer interface. This work proposes a real-time hand localization and recognition system. For hand localization, YOLOv3 is used that predicts a bounding box around the hand, and for hand gestures classification, a pretrained VGG16 network is employed. The bounding box regression technique helped localize the ROI (region of interest) and reduced the complexity, that aided in the classification task. The experimental results show that the proposed method is capable of recognizing the gestures with high testing accuracy on three benchmark datasets, namely, ASL (American sign language), Libras and NUS (National University of Singapore) datasets.