BEFM: A balanced and efficient fine-tuning model in class-incremental learning

Abstract

The ultimate objective of class-incremental learning (CIL) is to solve the stability–plasticity dilemma by continuously learning new courses without losing what has been learnt from the previous ones. Typical CIL methods tend to favor sample replay and parameter regularization, but recent studies have shown that exploiting and adapting historical models can effectively improve model performance. We present A Balanced and Efficient Fine-tuning Model (BEFM) to improve memory consumption efficiency and prevent catastrophic forgetting in CIL by utilizing historical models. In order to fully utilize various normalization techniques and preserve the model’s high plasticity and stability throughout training, we first create a multi-normalization module to replace the original single batch normalization procedure. On the other hand, we construct a model expansion method with knowledge distillation and offer a logit fine-tuning strategy to increase memory and computational efficiency. The model expansion strategy effectively solves the memory problem caused by feature aggregation by expanding only the deeper models with greater influence when learning a new task, while the knowledge distillation strategy encourages the model to retain the memory of the old task and improves the stability of the model. As a way to address the issue of class imbalance, the logit fine-tuning technique optimizes the standard softmax cross entropy, improving classifier design without adding computational burden. We test our method on the CIFAR10, CIFAR100, and miniImageNet100 datasets under various conditions. According to experimental results, our strategy outperforms current approaches with notable gains in performance. The code is available at https://github.com/Lize-Liu/BEFM-main.