Bayesian Kernel Regression for Noisy Inputs Based on Nadaraya-Watson Estimator Constructed from Noiseless Training Data

Abstract

In regression for noisy inputs, noise is typically removed from a given noisy input if possible, and then the resulting noise-free input is provided to the regression function. In some cases, however, there is no available time or method for removing noise. The regression method proposed in this paper determines a regression function for noisy inputs using the estimated posterior of their noise-free constituents with a nonparametric estimator for noiseless explanatory values, which is constructed from noiseless training data. In addition, a probabilistic generative model is presented for estimating the noise distribution. This enables us to determine the noise distribution parametrically from a single noisy input, using the distribution of the noise-free constituent of noisy input estimated from the training data set as a prior. Experiments conducted using artificial and real data sets show that the proposed method suppresses the overfitting of the regression function for noisy inputs and the root mean squared errors (RMSEs) of the predictions are smaller compared with those of an existing method.