Zero-Shot Sketch-Based Image Retrieval with teacher-guided and student-centered cross-modal bidirectional knowledge distillation

Abstract

In the context of zero-shot learning, the task of using unseen-class sketches as queries to retrieve real images is referred to as Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR). The ZS-SBIR task aims to generalize knowledge learned from known categories to unknown ones. Current research primarily relies on fine-tuning networks via loss functions or unidirectionally extracting knowledge from fixed-parameter teacher models for training student models. However, unidirectional knowledge extraction from teacher models often lacks mutual learning and knowledge alignment between the teacher and student models, while fine-tuning networks via loss functions struggles to handle both photo and sketch modalities simultaneously. Therefore, we designed a modal perception and distribution alignment scheme based on gradient weighting to explore both photo and sketch features bidirectionally and deeply investigate the relationships between different modalities. Building on this, we propose a teacher-guided and student-centered cross-modal bidirectional knowledge distillation framework. During training, the student and teacher models mutually learn discriminative information based on the relationships between different modalities and synchronize their parameters under the guidance of the teacher model, thus effectively achieving cross-modal alignment. Extensive experiments conducted on the TU-Berlin Ext, Sketchy Ext and QuickDraw Ext datasets demonstrate that our method significantly enhances retrieval performance.