GPTGAN: Utilizing the GPT language model and GAN to enhance adversarial text generation

Abstract

Training generative models that can generate high-quality and diverse text remains a significant challenge in the field of natural language generation (NLG). Recently, the emergence of large language models (LLMs) like GPT has enabled the generation of text with remarkable quality and diversity. However, building these models from scratch is both time-consuming and resource-intensive, making their comprehensive training practically unfeasible. Nonetheless, LLMs utility extends to addressing issues in other models. For instance, generative adversarial models often grapple with the well-known problem of mode collapse during training, leading to a trade-off between text quality and diversity. This means that these models tend to favor quality over diversity. In this study, we introduce a novel approach designed to enhance adversarial text generation by striking a balance between the quality and diversity of generated text, leveraging the capabilities of the GPT language model and other LLMs. To achieve this, we propose an enhanced generator that is guided by the GPT model. Essentially, the GPT model functions as a mentor to the generator, influencing its outputs. To achieve this guidance, we employ discriminators of varying scales on both real data and the texts generated by GPT. Experimental results underscore a substantial enhancement in the quality and diversity of outcomes across two benchmark datasets. Also the results demonstrate the generator’s ability to assimilate the output domain of the GPT language model. Furthermore, the proposed model exhibits superior performance in human evaluations when compared to other existing adversarial methods.