Gesture Detection using Tensor flow lite Efficient Net Model for Communication and E-learning Module for Mute and Deaf

Abstract

Human communication plays a vital role; without communicating, day-to-day tasks seem difficult to complete. And the world has an almost 5% population that struggles with hearing or speaking disability, which contributes to 430 million people worldwide, and this will grow up to 900million just in the next 25 to 30 years. With the increasing noise pollution, hearing capacity degrades, leading to various hearing problems. The WHO statistics show that 32million kids are acoustically impaired. With disabilities, there are multiple issues these people face, such as lack of learning facilities, job opportunities, communication platforms, etc. These people need a cooperative environment to express, learn at their pace and level of understanding. This paper focuses on developing an application that bridges the gap between these acoustically disabled people and people unknown to their way of communication. The proposed research is an edge device application provides features like a gesture to text, speech to text, e-learning platform, and Alert mechanism. This paper majorly focuses on developing a friendly all in one platform for mute and deaf community for communication, learning and emergency alerts. The research was conducted with two approaches the traditional CNN and Tensorflow lite Efficient Net model to train the ASL (American Sign Language) dataset for the communication platform, where we obtained accuracy of 98.91% and 98.82% respectively. To overcome the computational barriers of traditional CNN approach, Tensorflow lite Efficient Net model was brought into the picture. The proposed methodology would help build a platform for the deaf and mute community to express themselves better and gain wider exposure to the world.