EMF-GAN:Efficient Multilayer Fusion GAN for text-to-image synthesis

Abstract

Text-to-image generation is a challenging and significant research task. It aims to synthesize high-quality images that match the given descriptive statements. Existing methods still have problems in generating semantic information fusion insufficiently, and the generated images cannot represent the descriptive statements properly. Therefore, A novel method named EMF-GAN (Efficient Multilayer Fusion Generative Adversarial Network) is proposed. It uses a Multilayer Fusion Module (MF Module) and Efficient Multi-Scale Attention Module (EMA Module) to fuse the semantic information into the feature maps gradually. It realizes the full utilization of the semantic information and obtains high-quality realistic images. Extensive experimental results show that our EMF-GAN is highly competitive in image generation quality and semantic consistency. Compared with the state-of-the-art methods, EMF-GAN shows significant performance improvement on both CUB (FID from 14.81 to 10.74) and COCO (FID from 19.32 to 16.86) datasets. It can generate photorealistic images with richer details and text-image consistency. Code can be found at https://github.com/zxcnmmmmm/EMF-GAN-master.