A Novel Deep Generative Model via Semantic-Based Knowledge Distillation for Zero-Shot Learning

Abstract

Zero-Shot Learning (ZSL) aims to identify unseen target classes that lack training data. Most existing methods address the ZSL problem by generating samples of unseen classes based on the training data of seen classes and the semantic representations of unseen classes. However, due to the inherent limitations of ZSL, the generated unseen samples tend to be biased towards the data of seen classes, resulting in a label shift problem in the model’s projection domain. To address these issues, we propose a novel generation-based ZSL approach that incorporates semantic-based constraints and knowledge distillation. Specifically, the semantic regularization and preservation constraints are designed to improve the distribution and discriminability of the generated unseen data, respectively. Furthermore, the semantic-based knowledge distillation strategy is introduced to enhance the generative model’s feature encoding ability, thereby improving the quality of the generated unseen data. Extensive experiments on two standard ZSL benchmark datasets demonstrate that the proposed model achieves superior performance on both traditional and generalized ZSL tasks.