SF-GAN: Semantic fusion generative adversarial networks for text-to-image synthesis

Abstract

Text-to-image synthesis aims to generate high-quality realistic images conditioned on text description. The major challenge of this task rests on the deep and seamless integration of text and image features. Therefore, in this paper, we present a novel approach, e.g., semantic fusion generative adversarial networks (SF-GAN), for fine-grained text-to-image generation, which enables efficient semantic interactions. Specifically, our proposed SF-GAN leverages a novel recurrent semantic fusion network to seamlessly manipulate the global allocation of text information across discrete fusion blocks. Moreover, with the usage of the contrastive loss and the dynamic convolution, SF-GAN could fuse the text and image information more accurately and further improve the semantic consistency in the generate stage. During the discrimination stage, we introduce a word-level discriminator designed to offer the generator precise feedback pertaining to each individual word. When compared to current state-of-the-art techniques, our SF-GAN demonstrates remarkable efficiency in generating realistic and text-aligned images, outperforming its contemporaries on challenging benchmark datasets.