Indian Sign Language (ISL) Recognition And Translation using Mediapipe and LSTM

A person having hearing-impairment is always difficult to communicate with. Without a doubt, sign language has become the finest alternative and a very powerful tool for those who have difficulty speaking or hearing to communicate their opinions and emotions to others. It makes the process of integrating them into society easier and more convenient. The signs usually get jumbled and incomprehensible for someone who has never studied sign language or perceives it in another language. The goal is to develop a method that can enable two-way communication between hearing-impaired persons and others and to break down the communication barrier among those who don’t comprehend sign language and persons with impairments. The objective of this research is to demonstrate and create a method for understanding Indian sign language that combines MediaPipe for hand landmark extraction, LSTM for gesture training and recognition, and also captures either English or Hindi audio through the microphone and recognize it using Google Speech API or convert entered text into their relevant Indian sign language videos.