Evaluating machine translation systems for Brazilian sign language in the treatment of critical grammatical aspects

Machine translation tools for sign languages are developed with the aim of reducing barriers to access to information for deaf people. However, these technologies have some limitations related to the difficulty of dealing with some specific grammatical aspects of sign languages, which can negatively influence the experience of deaf users. In order to analyze this problem, in this study, we assess the translation of Brazilian Portuguese (Pt-br) content into Brazilian Sign Language (Libras) by three machine translators: HandTalk, Rybená and VLibras. More specifically, we conducted an experiment with 30 Brazilian human interpreters that evaluate the treatment of 7 specific grammatical aspects in the applications. As a result, we observed significant important limitation in the treatment of homonyms, adverbs (negation, mode and intensity), directional verbs, and phrases (interrogative and exclamatory) in the translations performed by these applications, indicating the need for them to improve the treatment of these grammatical aspects.