Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes

Abstract

Gaussian process regression is a widely applied method for function approximation and uncertainty quantification. The technique has recently gained popularity in the machine learning community due to its robustness and interpretability. The mathematical methods we discuss in this paper are an extension of the Gaussian process framework. We are proposing advanced kernel designs that only allow for functions with certain desirable characteristics to be elements of the reproducing kernel Hilbert space (RKHS) that underlies all kernel methods and serves as the sample space for Gaussian process regression. These desirable characteristics reflect the underlying physics; two obvious examples are symmetry and periodicity constraints. In addition, we want to draw attention to nonstationary kernel designs that can be defined in the same framework to yield flexible multitask Gaussian processes. We will show the impact of advanced kernel designs on Gaussian processes using several synthetic and two scientific data sets. The results show that informing a Gaussian process of domain knowledge, combined with additional flexibility and communicated through advanced kernel designs, has a significant impact on the accuracy and relevance of the function approximation.