Indian Sign Language Recognition using Deep Learning

Sign language is used by people having speaking and hearing disabilities. It generally has a set of words, where each word is represented by one or more hand gestures in sequence and may contain facial expressions. In order to address the interpretation/translation from sign language to English Language, we present our sign recognition approach for Indian sign language which aims to provide a method for interpreting signs in Indian sign language to words in English language translation. The approach is to have a vision based system in which the sequence of images representing a word in ISL is translated to equivalent English word. The translation would be done by means of Deep learning algorithms namely convolutional neural nets and recurrent neural nets. The system will be analyzing sequence of images, hence CNNs will analyze each image and their sequence is analyzed by LSTM (which is an implementation of RNN). We divided dataset into training dataset and testing dataset, which obtained 73.60% accuracy. The image distributions are kept fairly different in training and testing datasets.