A Chinese Sign Language Recognition System Using Leap Motion

Abstract

As an application of Gesture Recognition, Sign Language Recognition (SLR) is able to facilitate those who want to learn a manual communication system and makes it practicable for the majority of normal people, who are ignorant of sign language, to communicate with the group suffering from dysaudia and vice versa. In this paper, focusing on feasibility, we present a Chinese Sign Language (CSL) recognition system using the portable and cost-affordable Leap Motion sensor and applying kth-Nearest Neighbor (k-NN). Simultaneously, by introducing text to speech, speech synthesis and speech recognition (speech to text) APIs, we bring out a novel design of two-way communication system between normal people and hearing-impaired communities. The experiment result shows that such a CSL recognition system, which currently achieved static sign language interpretation with high accuracy, is able to play a much more significant role in this field.