Indian Sign Language Numeral Recognition Using Region of Interest Convolutional Neural Network

Abstract

Communication provide interaction among the people to exchange the feelings and ideas. The deaf community suffer a lot to interact with the community. Sign language is the way through which the people communicate with each other. In order to provide interaction with normal people there is a system which can convert the sign languages to the understandable form. The purpose of this work is to provide a real-time system which can convert Indian Sign Language (ISL) to the text. Most of the work based on handcrafted feature. In this we are introducing a deep learning approach which can classify the sign using the convolutional neural network. In the first phase we make a classifier model using the numeral signs using the Keras implementation of convolutional neural network using python. In phase two another real-time system which used skin segmentation to find the Region of Interest in the frame which shows the bounding box. The segmented region is feed to the classifier model to predict the sign. The system has attained an accuracy of 99.56% for the same subject and 97.26% in the low light condition. The classifier found to be improving with different background and the angle of the image captured. Our method focus on the RGB camera system.