Balanced ID-OOD tradeoff transfer makes query based detectors good few shot learners

Abstract

Fine-tuning is a popular approach to solve the few-shot object detection problem. In this paper, we attempt to introduce a new perspective on it. We formulate the few-shot novel tasks as a type of distribution shifted from its ground-truth distribution. We introduce the concept of imaginary placeholder masks to show that this distribution shift is essentially a composite of in-distribution (ID) and out-of-distribution(OOD) shifts. Our empirical investigation results show that it is significant to balance the trade-off between adapting to the available few-shot distribution and keeping the distribution-shift robustness of the pre-trained model. We explore improvements in the few-shot fine-tuning transfer in the few-shot object detection (FSOD) settings from three aspects. First, we explore the LinearProbe-Finetuning (LP-FT) technique to balance this trade-off to mitigate the feature distortion problem. Second, we explore the effectiveness of utilizing the protection freezing strategy for query-based object detectors to keep their OOD robustness. Third, we try to utilize ensembling methods to circumvent the feature distortion. All these techniques are integrated into a whole method called BIOT (Balanced ID-OOD Transfer). Evaluation results show that our method is simple yet effective and general to tap the FSOD potential of query-based object detectors. It outperforms the current SOTA method in many FSOD settings and has a promising scaling capability.