Emotional Paraphrasing Using Pre-trained Language Models

Emotion style transfer is a recent and challenging problem in Natural Language Processing (NLP). Transformer-based language models are becoming extremely powerful, so one wonders if it would be possible to leverage them to perform emotion style transfer. So far, previous work has not used transformer-based models for this task. To address this task, we fine-tune a GPT-2 model with corrupted emotional data. This will train the model to increase the emotional intensity of the input sentence. Coupled with a paraphrasing model, we develop a system capable of transferring an emotion into a paraphrase. We conducted a qualitative study with human judges, as well as a quantitative evaluation. Although the paraphrase metrics show poor performance compared to the state of the art, the transfer of emotion proved to be effective, especially for the emotions fear, sadness, and disgust. The perception of these emotions were improved both in the automatic and human evaluations. Such technology can significantly facilitate the automatic creation of training sentences for natural language understanding (NLU) systems, but it can also be integrated into an emotional or empathic dialogue architecture.