Domain adaptive YOLO based on image style selection and synergistic domain classifier

Abstract

Object detectors are trained on routine datasets that are primarily obtained under suitable conditions, yet will encounter various extreme environments in the complex real-world. Distribution shift in the train and test datasets poses serious damage to the performance of models, the most cost-effective means of solving this problem is unsupervised domain adaptive (UDA) method. In this work, we use YOLOv8 as underlying detector to construct a domain adaptive framework called YOLO-SDCoN, which offers a new solution paradigm for the domain shift problem. Specifically, we propose an Synergistic Domain Classifier (SDC) with richer gradient flow, which takes all the multi-scale features used for detection as inputs, providing a more adequate way to generate domain-invariant features while eliminating the gradient vanishing phenomenon. Furthermore, a novel Batch-Instance Co-Normalization (BI-CoN) method is proposed, which enables adaptive selection and preservation of image styles under the implicit guidance of a domain classifier, thereby generating better domain-invariant features to enhance the robustness of cross-domain detection. We conducted extensive experiments on KITTI, Cityscapes, Foggy Cityscapes, and SIM10K datasets. The results show that the proposed YOLO-SDCoN is comprehensively superior to the Faster R-CNN based domain adaptive frameworks, and achieves superior results compared to other methods.