Mongolian Sign Language Recognition Model

Sign language is a gesture-based manual used by people with hearing impairment and spoken language disorder to communicate with others. There is no universal sign language — the most used one is the American Sign Language. Mongolian Sign Language (MSL) has hand signs for letters of the alphabet, numbers, and other commonly used words. There are an estimated 16000 MSL signers. The lack of means to translate MSL into the Mongolian language, such as professional interpreters or translator applications, hinders MSL signers’ freedom of expression and political and public participation. Here, we created an MSL recognition system model that uses a camera to capture the letter symbols for the MSL alphabet and translates them into written Mongolian words. The proposed model uses two machine learning models that 1) recognize input, sorts, and filters, and 2) process Mongolian language. The model had an F1 score of 0.8678, given 51 distinct hand gestures. The natural language processing model that forms words had sufficient performance, though it can be improved in further works.