Recent Developments on Statistical Transfer Learning

Abstract

In recent years, there has been a significant increase in scholarly research on transfer learning. This review provides an in-depth examination of statistical transfer learning, categorising challenges into two domains: model-based and data distribution-based transfer learning. For model-based transfer learning, we assume sufficient similarity between the source and target domain models, which allows the effective use of source data for target tasks. This framework applies to various problems, including (generalised) linear regression, quantile regression, kernel regression, functional regression and graphical model. From another point of view, data distribution-based transfer learning addresses variations due to covariate shift and posterior drift, enhancing the reliability of shared data from diverse sources. The primary objective is to introduce typical methods and data structures adapted for transfer learning, along with resolution techniques. Additionally, we elucidate theoretical concepts such as consistency and minimax optimality to provide a comprehensive understanding of the subject. Furthermore, we explore other related topics in statistical transfer learning that tackle various problems.