Long Short-Term Memory-based Static and Dynamic Filipino Sign Language Recognition

Abstract

Filipino Sign Language (FSL) is a distinctive form of sign language with its own set of pose, gestures, and grammar, which can cause a challenge in terms of identifying it. Earlier researches have identified three categories of sign language recognition methods, these are the gloves-based, vision-based and hybrid. Existing studies are only limited to recognizing static FSL and only include limited phrases or words for dynamic FSL recognition. Since the existing studies for dynamic FSL are only capable of recognizing limited words and phrases, this could limit the communication. Thus, adding more phrases or words for dynamic FSL recognition is significant. The objective of this study is to create a FSL recognition model using Long Short-term Memory and MediapPipe Holistic pipeline. A recurrent neural network type called Long Short-Term Memory (LSTM) is capable of recognizing sign language gestures, including FSL, because it can handle long-term dependencies. Using 11,070 sequences, this study trained a model to recognize 24 static Filipino sign languages including alphabets and 17 dynamic Filipino sign languages including common Filipino words, greetings, and phrases. Since a lot of people in the Philippines are not familiar with FSL, this study is useful to improve communication, and making it easier for people who do not understand FSL to understand Filipino deaf or speech impaired people in a certain communication. The recognition model produced by the proponents achieved a 99.72% model accuracy score using MediaPipe and LSTM, and can accurately detect and interpret static and dynamic Filipino sign language gestures in real-time.