Enhancing sentiment and emotion translation of review text through MLM knowledge integration in NMT

Abstract

Producing a high-quality review translation is a multifaceted process. It goes beyond successful semantic transfer and requires conveying the original message’s tone and style in a way that resonates with the target audience, whether they are human readers or Natural Language Processing (NLP) applications. Capturing these subtle nuances of the review text demands a deeper understanding and better encoding of the source message. In order to achieve this goal, we explore the use of self-supervised masked language modeling (MLM) and a variant called polarity masked language modeling (p-MLM) as auxiliary tasks in a multi-learning setup. MLM is widely recognized for its ability to capture rich linguistic representations of the input and has been shown to achieve state-of-the-art accuracy in various language understanding tasks. Motivated by its effectiveness, in this paper we adopt joint learning, combining the neural machine translation (NMT) task with source polarity-masked language modeling within a shared embedding space to induce a deeper understanding of the emotional nuances of the text. We analyze the results and observe that our multi-task model indeed exhibits a better understanding of linguistic concepts like sentiment and emotion. Intriguingly, this is achieved even without explicit training on sentiment-annotated or domain-specific sentiment corpora. Our multi-task NMT model consistently improves the translation quality of affect sentences from diverse domains in three language pairs.