Unsupervised optimal deep transfer learning for classification under general conditional shift.

Abstract

Classifiers trained on labeled source data may yield misleading results when applied to unlabeled target data drawn from a different distribution. Transfer learning can rectify this by transferring knowledge from source to target data, but its effectiveness frequently relies on stringent assumptions, such as label shift or strong separation conditions. We introduce a novel general conditional shift assumption, which encompasses label shift as a special case and facilitates the identifiability of both the target distribution and the shift function without requiring a separation condition. Our classifier is constructed by integrating deep neural networks (DNNs) and a pseudo-maximum likelihood approach. We establish asymptotic error bounds for our DNN-based classifier and estimators of the conditional probabilities ${ \boldsymbol{\eta }_{P}}$ for source data and the target label distribution $\boldsymbol{\pi }_{Q}$, in terms of the intrinsic dimension of ${ \boldsymbol{\eta }_{P}}$. Notably, the excess risk of the proposed classifier achieves the optimal minimax rate, up to a logarithmic factor. Our method not only eliminates the need to estimate the shift function, but also alleviates the curse of dimensionality when ${ \boldsymbol{\eta }_{P}}$ exhibits a low-dimensional structure. Numerical simulations, along with an analysis of an Alzheimer's disease dataset, underscore its exceptional performance.