Heterogeneity-aware transfer learning for high-dimensional linear regression models

Abstract

Transfer learning can refine the performance of a target model through utilizing beneficial information from relevant source datasets. In practice, however, auxiliary samples may be collected from different sub-populations with non-negligible heterogeneity. In this paper we assume that each dataset involves a common parameter vector and dataset-specific nuisance parameters and extend the transfer learning framework to account for heterogeneous models. Specifically, we adapt the decorrelated score technique to deal with the dataset-specific nuisance parameters and develop a strategy to leverage possible shared information from relevant source datasets. To avoid negative transfer, a completely data-driven algorithm is provided to determine the transferable sources. The convergence rate of the proposed estimator is investigated and the source detection consistency is also verified. Extensive numerical experiments are conducted to evaluate the proposed transfer learning algorithms, and an application to the Genotype-Tissue Expression dataset is exhibited.