Multi-granularity contrastive zero-shot learning model based on attribute decomposition

Abstract

Zero-shot learning (ZSL) aims to identify new classes by transferring semantic knowledge from seen classes to unseen classes. However, existing models lack a differentiated understanding of different attributes and ignore the impact of global context information. Therefore, we propose a multi-granularity contrastive zero-shot learning model based on attribute decomposition. Specifically, as attributes are the carriers of semantic knowledge, we first classify attributes into key attributes and common attributes, i.e., attribute decomposition, and the importance of common attributes is increased by key attribute mask prediction. Then, inspired by Navon’s global–local paradigm, we work out the multi-granularity contrastive learning model, which is composed of the global learning module and the local one, to further enhance the interaction between the global and local information. Finally, zero-shot image classification is achieved by training a multi-granularity contrastive learning model. The method is experimented on three public ZSL benchmark datasets (i.e., AWA2, CUB, and SUN). Compared with the existing model, this model improves the accuracy by 2.2%/5.4% (AWA2/SUN) on conventional ZSL, 2.5%/1.6%/6.3% (AWA2/CUB/SUN) on generalized ZSL, further verifying the effectiveness of this model.