Dynamic VAEs via semantic-aligned matching for continual zero-shot learning

Abstract

Continual Zero-shot Learning (CZSL) is capable of classifying unseen categories across a sequence of tasks. However, CZSL is often plagued by the challenge of catastrophic forgetting. While recent studies have shown that preserving past data for experience replay can effectively address this issue, it may be limited to specific scenarios and pose a risk of data leakage. Additionally, many existing CZSL models fail to adequately highlight the correlation between semantic and visual features. To tackle these shortcomings, we introduce dynamic Variational Autoencoders (VAEs) via semantic-aligned matching for CZSL. The proposed model utilizes both semantic and visual VAEs to enhance the transfer capability of knowledge from past tasks. Leveraging generative experience replay, our model effectively combats catastrophic forgetting. Our approach was assessed on five datasets: aPY, AWA1, AWA2, CUB, and SUN, yielding superior performance to baseline models.