Integrating cycleGAN and BERT for Chinese text style transfer

Abstract

In this study, we integrate the Bidirectional Encoder Representations from Transformers (BERT) model with the Cycle Generative Adversarial Network (CycleGAN) to create a system for Chinese text style transfer. Natural language processing (NLP) involves converting human languages into data interpretable by computers, enabling applications like text classification, chatbots, and dialogue systems. Recent advancements, such as Google's transformer model and the BERT technique, have significantly improved NLP capabilities through self-attention mechanisms and unsupervised pretraining. Text style transfer modifies the style of texts without altering their semantics. Previous methods like StyIns and models based on disentangled representation learning highlight the challenges of retaining text meaning during style transfer. Our system leverages CycleGAN’s unsupervised learning to convert unpaired data between wuxia and fantasy styles while preserving semantics. Using the pretrained BERT model from the Chinese Knowledge and Information Processing (CKIP) Lab, our experimental results demonstrate successful style conversion, maintaining the original meanings of texts. This integration of BERT and CycleGAN shows promise for further advancements in NLP applications.