Semi-PKD: Semi-supervised Pseudoknowledge Distillation for saliency prediction

Abstract

In saliency prediction, Knowledge Distillation (KD) is leveraged to improve the predictive performance of compact Student Networks. However, the challenge is searching for an optimal teacher–student pair while handling the unavailability of large-scale annotations in the Pseudoknowledge Distillation (PKD). To overcome this challenge, a semi-supervised method is proposed; Semi-PKD. This method involves pseudo-label generation on unlabeled data by a Teacher Network trained using the exponential moving average KD (EMA-KD) method. The EMA-KD method utilizes only the Student Network by acquiring self-knowledge, solving the problem of optimal teacher–student pair selection. Semi-PKD outperforms other state-of-the-art saliency prediction models across various evaluation metrics. The code is available at https://github.com/chakkritte/Semi-PKD.