Neural Machine Translation Approach in Automatic Translations between Portuguese Language and Portuguese Sign Language Glosses

Abstract

Communication between deaf and non-deaf people is a difficult task in Portugal. Only a very small portion of the population is able to communicate with the impaired ones. Unfortunately, there aren’t a lot of systems which are available to the public to help in this communication. In this paper we present a component to translate from Portuguese Natural Language to Portuguese Sign Language, using Machine Learning approach, which is a sub-component of a bigger system designed to allow communication between deaf and non-deaf via smartphones. Our approach uses a Neural Machine Translation system to provide translations between Portuguese Natural and Portuguese Sign Language Glosses to be used to animate a 3D avatar. Early testing of different machine learning architectures showed promising results of BiLingual Evaluation Understudy (BLEU) evaluation on translations. The component uses a supervised learning model and results have shown that, under normal operating conditions an accuracy in the detection to translate from Portuguese-to-Portuguese sign language glosses, achieving above 80% on testing data using BLEU performance scores.