Sign Language to Text for Deaf and Dumb

Abstract

Hand-Sign Language Gestures are nonverbal messages that help in communication and can be understood with vision. Since the only disability Deaf & Dumb people have is communication related to speech therefore they cannot use spoken languages hence the only way for them to communicate with the people having same disability is through this hand-sign language. This causes a language barrier as normal people can not understand their hand sign language and vice a versa also since most of the people are not familiar with the same and interpreters are not very user-friendly they end up in a deadlock. Therefore, this paper proposes a CNN-based method for deciphering sign language and then converting it to text. In the proposed scheme of this paper the main focus is on fingerspelling and an additional feature of emotion recognition to support the interpretation with the 3rd component of sign language i.e non-manual features, a real-time solution for easy interpretation of sign language for normal human beings as well as Deaf & Dumb people using convolutional neural networks breaking the language barrier. The experimental results show that this paper achieved an accuracy score of 99.8% on testing data which is better than the majority of the recent research papers on American gesture-based communication