Gesture based Real-Time Sign Language Recognition System

Abstract

Real-Time Sign Language Recognition (RTSLG) can help people express clearer thoughts, speak in shorter sentences, and be more expressive to use declarative language. Hand gestures provide a wealth of information that persons with disabilities can use to communicate in a fundamental way and to complement communication for others. Since the hand gesture information is based on movement sequences, accurately detecting hand gestures in real-time is difficult. Hearing-impaired persons have difficulty interacting with others, resulting in a communication gap. The only way for them to communicate their ideas and feelings is to use hand signals, which are not understood by many people. As a result, in recent days, the hand gesture detection system has gained prominence. In this paper, the proposed design is of a deep learning model using Python, TensorFlow, OpenCV and Histogram Equalization that can be accessed from the web browser. The proposed RTSLG system uses image detection, computer vision, and neural network methodologies i.e. Convolution Neural Network to recognise the characteristics of the hand in video filmed by a web camera. To enhance the details of the images, an image processing technique called Histogram Equalization is performed. The accuracy obtained by the proposed system is 87.8%. Once the gesture is recognized and text output is displayed, the proposed RTSLG system makes use of gTTS (Google Text-to-Speech) library in order to convert the displayed text to audio for assisting the communication of speech and hearing-impaired person.