A novel probabilistic transfer learning strategy for polynomial regression

Abstract

In the field of surrogate modeling and, more recently, with machine learning, transfer learning methodologies have been proposed in which knowledge from a source task is transferred to a target task where sparse and/or noisy data result in an ill-posed calibration problem. Such sparsity can result from prohibitively expensive forward model simulations or simply lack of data from experiments. Transfer learning attempts to improve target model calibration by leveraging similarities between the source and target tasks.This often takes the form of parameter-based transfer, which exploits correlations between the parameters defining the source and target models in order to regularize the target task. The majority of these approaches are deterministic and do not account for uncertainty in the model parameters. In this work, we propose a novel probabilistic transfer learning methodology which transfers knowledge from the posterior distribution of source to the target Bayesian inverse problem using an approach inspired by data assimilation.While the methodology is presented generally, it is subsequently investigated in the context of polynomial regression and, more specifically, Polynomial Chaos Expansions which result in Gaussian posterior distributions in the case of iid Gaussian observation noise and conjugate Gaussian prior distributions. The strategy is evaluated using numerical investigations and applied to an engineering problem from the oil and gas industry.