Convolutional Neural Network Hand Gesture Recognition for American Sign Language

With the advancements in the computer vision technology, learning and using sign languages to communicate with deaf and mute people has become easier. Exciting research is ongoing for providing a global platform for communication in different sign languages. In this paper, we present a Deep Learning based approach to recognize a sign performed in American Sign Language by capturing an image as input. The system can predict the signs of 0 to 9 digits performed by the user. By utilizing image processing to convert RGB data to grayscale images, efficient reduction is achieved in the storage requirements and training time of the Convolutional Neural Network. The objective of the experiment is to find a mix of Image Processing and Deep Learning Architecture with lesser complexity to deploy the system in mobile applications or embedded single board computers. The database is trained from scratch using smaller networks as LeNet-5 and AlexNet as well as deeper network such as Vgg16 and MobileNet v2. The comparison of the recognition accuracies is discussed in the paper. The final selected architecture has only 10 layers including a dropout layer which boosted the training accuracy to 91.37% and testing accuracy to 87.5%.