Sequence to Sequence CycleGAN for Non-Parallel Sentiment Transfer with Identity Loss Pretraining

Abstract

Sentiment transfer has been explored as non-parallel transfer tasks in natural language processing. Previous works depend on a single encoder to disentangle either positive or negative style from its content and rely on a style representation to transfer the style attributes. Utilizing a single encoder to learn disentanglement in both styles might not sufficient due to the different characteristics of each sentiment represented by various vocabularies in the corresponding style. To this end, we propose a sequence to sequence CycleGAN which trains different text generators (encoder-decoder) for each style transfer direction. Learning disentangled latent representations leads previous works to high sentiment accuracy but suffer to preserve the content of the original sentences. In order to manage the content preservation, we pretrained our text generator as autoencoder using the identity loss. The model shows an improvement in sentiment accuracy and BLEU score which indicates better content preservation. It leads our model to a better overall performance compared to baselines.