Semantics-Conditioned Generative Zero-Shot Learning via Feature Refinement

Generative zero-shot learning (ZSL) recognizes novel categories by employing a cross-modal generative model conditioned on semantic factors (such as attributes) to transfer knowledge from seen classes to unseen ones. Many existing generative ZSL methods rely solely on feature extraction models pre-trained on ImageNet, disregarding the cross-dataset bias between ImageNet and ZSL benchmarks. This bias inevitably leads to suboptimal visual features that lack semantic relevance to the predefined attributes, constraining the generator’s ability to synthesize semantically meaningful visual features for generative ZSL. In this paper, we introduce a visual feature refinement method (ViFR) to mitigate cross-dataset bias and advance generative ZSL. Given a generative ZSL model, ViFR incorporates both pre-feature refinement (Pre-FR) and post-feature refinement (Post-FR) modules to simultaneously enhance visual features. In Pre-FR, ViFR aims to learn attribute localization for discriminative visual feature representations using an attribute-guided attention mechanism optimized with attribute-based cross-entropy loss. In Post-FR, ViFR learns an effective visual\(\rightarrow \)semantic mapping by integrating the semantic-conditioned generator into a unified generative model to enhance visual features. Additionally, we propose a self-adaptive margin center loss (SAMC-loss) that collaborates with semantic cycle-consistency loss to guide Post-FR in learning class- and semantically-relevant representations. The features in Post-FR are concatenated to form fully refined visual features for ZSL classification. Extensive experiments on benchmark datasets (i.e., CUB, SUN, and AWA2) demonstrate that ViFR outperforms state-of-the-art ZSL approaches. Our implementation is publicly available at https://github.com/shiming-chen/ViFR.