A Vision-based System for Recognition of Words used in Indian Sign Language Using MediaPipe

Abstract

Indian Sign Language (ISL) is a form of communication used in India by the speech and hearing impaired community. It conveys linguistic information through gestures of the hands, arms, face, and head. However, the gestures used may not always be directly related to the referent term, resulting in a significant communication gap. Hence there is a need for a translator that can translate ISL into text or speech. The proposed system aims to recognize signs of ISL and translate them into texts that can be easily read. The ISL recognition system is based on Google’s MediaPipe as a feature extractor and Random Forest Classifier is used for classification. An accuracy of 97.4% is achieved. The results show that the integration of MediaPipe with ML algorithms may be effectively employed to correctly recognise signs of ISL.