Deep Learning with OBH for Real-Time Rotation-Invariant Signs Detection

Abstract

Numerous studies are being undertaken to provide answers for sign language recognition and classification. Deep learning-based models have higher accuracy (90%-98%); however, require more runtime memory and processing in terms of both computational power and execution time (1 hour 20 minutes) for feature extraction and training images. Besides, deep learning models are not entirely insensitive to translation, rotation, and scaling; unless the training data includes rotated, translated, or scaled signs. However, Orientation-Based Hashcode (OBH) completes gesture recognition in a significantly shorter length of time (5 minutes) and with reasonable accuracy (80%-85%). In addition, OBH is not affected by translation, rotation, scaling, or occlusion. As a result, a new intermediary model is developed to detect sign language and perform classification with a reasonable processing time (6 minutes) like OBH while providing attractive accuracy (90%-96%) and invariance qualities. This paper presents a coupled and completely networked autonomous system comprised of OBH and Gabor features with machine learning models. The proposed model is evaluated with 576 sign alphabet images (RGB and Depth) from 24 distinct categories, and the results are compared to those obtained using traditional machine learning methodologies. The proposed methodology is 95.8% accurate against a randomly selected test dataset and 93.85% accurate after 9-fold validation.