DFS-GAN: A One-Stage Backbone Enhancement Model for Text-to-Image

Abstract

The text-to-image technology primarily relies on generative adversarial networks (GANs). However, traditional GANs encounter several challenges, for example, limited semantic correlation between generated images and textual information, fuzzy details and inadequate structural integrity, and the prevalent utilisation of redundant phased network architectures. In this paper, we propose a deep fusion generative adversarial network (DF-GAN) enhancement model (DFS-GAN) combined with a self-attention mechanism. The generator of the DF-GAN model is more streamlined compared to previous network models, enabling it to synthesise images with higher quality and text-image semantic consistency. We also made targeted improvements based on the “limitations” mentioned in the DF-GAN paper, specifically addressing the model's ability to synthesise fine-grained features and the use of existing pre-trained large models. bidirectional encoder representations from transformers (BERT) is used to mine the semantic features of text context, and the deep text-image fusion block (DFBlock) is added to realise the matching of deep text semantics and image regional features. Then, a self-attention mechanism module is introduced as a supplement to the convolution module at the model architecture level, aiming to better establish long-distance and multi-level dependencies. The experimental results show that the proposed DFS-GAN model not only strengthens the semantic relationship between the text and the image but also ensures the precise details and overall integrity of the generated image.