Fuzzy Domain Adaptation From Heterogeneous Source Teacher Models

Abstract

Unsupervised domain adaptation that relies on data matching can raise privacy concerns when leveraging transferable knowledge from the source domain(s). To address this issue, source-free domain adaptation has been proposed and subsequently developed. However, privacy risks persist due to potential data leaks from model inversion attacks even when using only source models without accessing the source data directly. Alongside this, another underexplored problem is feature heterogeneity across multiple source domains. In this article, we propose a fuzzy domain adaptation method, fuzzy heterogeneous domain adaptation (FuzHDA), that learns fuzzy rules from heterogeneous teacher models, even when these models are black boxes. The proposed method first trains heterogeneous source models privately on individual devices without sharing any information, including neither data nor model parameters. Then, using a memory bank that stores all target predictions from the pretrained source models, we apply a self-knowledge distillation approach to train a target model, which simulates the predictions from these heterogeneous source teachers. After completing the target model, a cross-modality hybrid model leveraging prompt learning and uncovering causal factors is fine-tuned via self-supervision at the last step to facilitate transfer performance. Experiments on real-world datasets demonstrate the superiority of the proposed FuzHDA.