Low Resolution Hand Gestures Recognition of Bengali Sign Alphabet by Using a Convolutional Neural Network

Abstract

Sign language is an essential tool for the deaf and the hard of hearing community of approximately 1.33 billion people. Due to this fact, researches have been conducted for decades for near-accurate recognition of sign characters. Sensor-based approaches and vision-based approaches have been adapted so far for tackling this dilemma. Sensor-based approaches can obtain high performance but it is costly and demands physical contact to sensors. On the other hand, vision-based approaches are not costly, need no contact but have not yet been able to produce a high accuracy like sensor-based approaches. The dilemma of sign characters recognition gets more problematic for Bengali sign language as not many datasets regarding Bengali sign language are available. Moreover, not many significant contributions can be found in this domain like other popular languages such as English, Turkish, Japanese, and Indian sign language. Furthermore, one of the most popular Bengali sign language datasets, Ishara-Lipi, consists of a few low-resolution samples. This study is focused on recognizing the low-resolution hand gestures of Bengali sign language. In this research, a convolutional neural network has been proposed which is suitable for the recognition of low-resolution sign gestures. Experimental results showed that the proposed approach achieved 99.08%, 99.38%, and 99.07% overall accuracy for digits, characters, and both digits and characters of the Ishara-Lipi dataset respectively.