Learning Unsupervised Side Information for Zero-Shot Learning

Zero-Shot Learning aims to recognize unseen class images that do not appear in training, which is attracting more and more research interests in recently years. Side information is an important key to ZSL since it transfers the knowledge between seen and unseen classes. Human annotated attribute, as the most popular side information, need much human effort and time consumption during data collection. While unsupervised side information such as word2vec is not performing well since they lack the representation ability for visual information. In this paper, we propose to use CLIP features, which is learned with image and natural language pairs without human efforts, to perform ZSL. Extensive experiments on two benchmark datasets, AWA2 and CUB, demonstrates that our method is achieving impressive accuracy gain over word2vec, even beats human attributes in some circumstances.