Class-conditional image synthesis with intra-class relation preservation

Abstract

Modeling class-conditional data distributions remains challenging, since the intra-class variation may be very large. Different from generic class-conditional Generative Adversarial Networks (GANs), we take inspiration from the observation that there may exist multiple modes with diverse visual appearances in a single class, and propose an Intra-class Prototype-based Relation Preservation (IPRP) approach to improve class-conditional image synthesis. Toward this end, a generator is designed to learn class-specific data distribution, conditioned on intra-class prototype-based relation. To associate label embeddings with the cluster prototypes, we incorporate an auxiliary prototypical network to perform adversarial interpolation, and the synthesized data are required to encapsulate their relation to the corresponding prototypes in the form of interpolation coefficients. The prototypical network can be further leveraged to improve the class-conditional real-fake identification performance by injecting semantics-aware features into a discriminator. This design allows the generator to better capture intra-class modes We conduct extensive experiments to demonstrate that IPRP outperforms the competing class-conditional GANs in terms of data diversity and semantic accuracy.