Interpretation of Hand Spelled Banking Helpdesk Terms for Deaf and Dumb Using Deep Learning

Abstract

The hand sign language uses visual-manual modality to share a certain message. Specially abled people having hearing and speaking disabilities interact more naturally in hand sign language rather than verbal language. According to one of the Census study, 2.21% out of 121 crore population in India are ‘disabled’, out of which 19% are having a hearing disability and 7% are having speech disability. Since everyone cannot communicate in sign language as it is a lesser-known language, it often leads to communication gap. So, the automated Sign Language Interpreter (SLI) helps to meet this communication gap as a manual sign language translator is not a convenient option because of its privacy issues and lack of availability. This paper proposes an Indian Hand Sign Language Interpreter which operates upon a vision-based approach that uses Machine Learning and Deep Learning techniques to locate the hand gesture region accurately for extracting the features and finally interpreting the respective meaning. The experimentation for performance metrics such as accuracy and loss using various activation functions helped to analyzed the performance of the model. The system successfully identifies a number of hand spelled words and thus eases the communication among people.