Real-time Mexican Sign Language recognition

Abstract

The Mexican Sign Language (MSL) has its own structure and grammar different from the Spanish spoken in Mexico. The MSL is composed of more than 1000 signs used by the deaf community of this country. Few hearing people know and use MSL, therefore there is a communication barrier between hearing and deaf people. In order to overcome this issue, we propose a non-invasive sign detection application, so that the interlocutor interacts with the system in a natural way interpreting a set of signs of the MSL using a bimodal cognitive vision system. Our application uses a Kinect sensor to obtain the gestures, geometric moments and Hidden Markov models to process it. From experiments performed with 10 people, it was concluded that our application achieved an average sensitivity of 86% and an average specificity of 80%.