HAND SIGN LANGUAGE RECOGNITION USING AUGMENTED REALITY & MACHINE LEARNING

Abstract

Effective communication is a cornerstone of human interaction, fostering societal cohesion and development. Throughout history, communication has evolved from primitive drawings to complex languages, shaping our societys fabric. However, amidst this progression, individuals with speech and hearing impairments have often faced significant challenges in communication. Despite constituting a minority, their needs are paramount and must not be overlooked. Recognizing the diverse classification of languages into verbal and non-verbal forms, it becomes evident that non-verbal languages play a crucial role, especially for Individuals with Hearing and Speech Impairments (IWSHI). These individuals rely on non-verbal communication methods to interact with the world around them, yet they often face barriers due to the lack of understanding and accessibility. To address this challenge, the HSLR app serves as a transformative tool, enabling IWSHI to communicate confidently. Leveraging technologies such as Augmented Reality (AR) and Machine Learning (ML), our app facilitates real-time recognition of hand signs, providing instantaneous translations for seamless communication. Additionally, the integration of AR technology enhances the user experience, offering immersive and interactive sign-language communication platforms. The MediaPipe model used in real-time achieves high accuracy in recognizing sign language due to the ample dataset we provided. KEY WORDS: Hand Sign Language Recognition (HSLR), Augmented Reality (AR), Machine Learning (ML), American Sign Language (ASL), Computer Vision, MediaPipe