Potential Knowledge Extraction Network for Class-Incremental Learning

Abstract

Class-Incremental Learning (CIL) aims to dynamically learn new classes without forgetting the old ones, and it is typically achieved by extracting knowledge from old data and continuously transferring it to new tasks. In the replay-based approaches, selecting appropriate exemplars is of great importance since exemplars represent the most direct form of retaining old knowledge. In this paper, we propose a novel CIL framework: Potential Knowledge Extraction Network (PKENet), which addresses the issue of neglecting the knowledge of inter-sample relation in most existing works and suggests an innovative approach for exemplar selection. Specifically, to address the challenge of knowledge transfer, we design a relation consistency loss and a hybrid cross-entropy loss, where the former works by extracting structural knowledge from the old model while the latter captures graph-wise knowledge, enabling the new model to acquire more old knowledge. To enhance the anti-forgetting effect of exemplar set, we devise a maximum-forgetting-priority method for selecting samples most susceptible to interference from the model’s update. To overcome the prediction bias problem in CIL, we introduce the Total Direct Effect inference method into our model. Experimental results on CIFAR100, ImageNet-Full and ImageNet-Subset datasets show that multiple state-of-the-art CIL methods can be directly combined with our PKENet to reap significant performance improvement. Code: https://github.com/XXDyeah/PKENet.