Vision transformer-based generalized zero-shot learning with data criticizing

Abstract

Generalized Zero-Shot Learning (GZSL) aims to enable accurate testing and recognition of unseen classes by utilizing training data from seen classes and leveraging attribute knowledge. However, GZSL faces a challenge wherein the model, trained solely on seen class data, tends to be biased towards recognizing visual features of seen classes, resulting in poorer recognition performance for unseen classes. To address this issue, we propose an approach called Vision Transformer-Based Generalized Zero-Shot Learning with Data Criticizing (ViT-DaCr). In order to obtain improved visual features, we thoroughly examine features extracted by Vision Transformer (ViT) with a new design. Additionally, we recognize that not all training data align with our model during the training process, leading the model to exhibit a bias towards recognizing visual features of seen classes and directly impacting visual feature recognition. Therefore, we propose a data critic mechanism that utilizes Adjusted Boxplot to filter out such data automatically during the training process. Extensive experiments demonstrate the advanced performance of our model on three challenging and popular datasets.