ML-GAN: Multi-level Text-driven Fine-grained Image Generation using Generative Adversarial Network

Text-to-image generation aims to convert input text into semantically accurate and visually realistic images. Existing methods typically generate a general outline of an image using sentence-level text and then refine it with word-level text. However, this approach of jumping directly from coarse-grained to fine-grained text features makes it challenging for the model to refine images accurately. To address this issue, we propose a Multi-level Text-driven Fine-grained Image Generation using Generative Adversarial Networks (ML-GAN). The model leverages different hierarchical levels of text information to construct and optimize generated images progressively. We design a Dual-level Text Parallel Fusion Module (DPFM) and a Triple-level Text Parallel Fusion Module (TPFM) to precisely adjust and optimize image details by utilizing text information at different levels. Additionally, to enhance semantic consistency between text and generated images, we introduce a Cross-modal Attention Fusion Module in the discriminator to improve its ability to recognize text-image matching, thereby guiding the generator to produce images that better match text content. Compared to baseline models, our proposed model achieves improvements of 19.10% and 12.99% in FID scores on the CUB and COCO datasets, respectively. This validates the effectiveness of the multi-level text-to-image transformation approach in enhancing the quality of generated images.