High-Quality-High-Quantity Semantic Distillation for Incremental Object Detection

Abstract

Model is required to learn from dynamic data stream under incremental object detection task. However, traditional object detection model fails to deal with this scenario. Fine-tuning on new task suffers from a fast performance decay of early learned tasks, which is known as catastrophic forgetting. A promising way to alleviate catastrophic forgetting is knowledge distillation, which includes feature distillation and response distillation. Previous feature distillation methods have not discuss knowledge selection and knowledge transfer at the same time. In this paper, we propose high-level semantic feature distillation and task re-balancing strategy that consider both high-quality knowledge selection and high-quantity knowledge transfer simultaneously. Extensive experiments are conducted on MS COCO benchmarks. The performance of our method exceeds previous SOTA methods under all experimental scenarios. Remarkably, our method reduces the mAP gap toward full-training to 2.58, which is much better than that of the previous SOTA method with a gap of 3.30.