Design of a neural transformer for Spanish to Mexican Sign Language automatic translation/interpretation.

Abstract

This paper uses a multi-head neural transformer to present the text-to-text translation/interpretation of Sign Language (SL) in the context of glosses (written SL). A Spanish to Mexican Sign Language (MSL) gloss dataset was built based on simple and compound sentences and the corresponding interpretation in MSL gloss. The interpretation process was achieved by implementing state-of-the-art tools in the natural language processing (NLP) field called neural transformers. We tried different architectures, varying the number of encoder-decoder layers and hyperparameters. The best of our models achieved 0.68 BLEU in the training phase and 0.33 in the validation phase. MSL glosses are crucial as they rule the grammatical order in which MSL has to be executed. All these quantitative and qualitative results confirm the potential applicability of neural transformers to create effective automatic translators for the Spanish language to MSL, with similar effectiveness shown by other automatic translators for other more likely languages.