Dual transferable knowledge interaction for source-free domain adaptation

Abstract

Source-free domain adaptation (SFDA) aims to adapt a pre-trained model to an unlabeled target domain without requiring access to source data, addressing privacy and security concerns in real-world applications. While vision-language models like CLIP have shown promise for SFDA, existing approaches primarily leverage CLIP’s final predictions for adaptation, overlooking its feature-space discriminative insights. This limitation hinders knowledge transfer effectiveness. To bridge this gap, we propose Dual Transferable Knowledge Interaction (DTKI), a novel framework that integrates local feature structures from CLIP with inter-class relationships from the source model to guide adaptation. Specifically, DTKI constructs a nearest-neighbor graph to capture local target domain structures and enhances CLIP’s textual representations using inter-class relationships from the source model’s classifier. Our theoretical analysis demonstrates that these two complementary knowledge transfer mechanisms significantly reduce classification errors. Extensive experiments on four public SFDA benchmarks validate DTKI’s superiority, achieving state-of-the-art performance across multiple domain adaptation scenarios, including partial-set, closed-set, and open-set SFDA.