Semi-Supervised Learning with Gaussian Processes

Abstract

As a supervised learning algorithm, the standard Gaussian processes has the excellent performance of classification. In this paper, we present a semi-supervised algorithm to learning a Gaussian process classifier, which incorporating a graph-based construction of semi-supervised kernels in the presence of labeled and unlabeled data, and expanding the standard Gaussian processes algorithm into the semi-supervised learning framework. Our algorithm adopts the spectral decomposition to obtain the kernel matrices, and employs a convex optimization method to learn an optimal semi-supervised kernel, which is incorporated into the Gaussian process model. In the Gaussian processes classification, the expectation propagation algorithm is applied to approximate the Gaussian posterior distribution. The main characteristic of the proposed algorithm is that we incorporate the geometric properties of unlabeled data by globally defined kernel functions. The semi-supervised Gaussian processes model has an explicitly probabilistic interpretation, and can model the uncertainty among the data and solve the complex non-linear inference problems. In the presence of few labeled examples, the proposed algorithm outperforms cross-validation methods, and we present the experimental results demonstrating the effectiveness of this algorithm in comparison with other related works in the literature.