Regression Method for Noisy Inputs Based on Non-Parametric Estimator Constructed from Noiseless Training Data

Abstract

The regression method proposed in this paper determines a regression function for noisy inputs. We represent noisy inputs by using noise and latent noise-free constituent of the noisy input. Given an observed noisy input, the proposed method estimates the posterior of the latent noise-free constituent of it, and represents the posterior using the noise distribution. For the value of the regression function for the noisy input, the method produces the expected value of the Nadaraya–Watson estimator for noiseless inputs, which is constructed from a training dataset consisting of noiseless explanatory values and the corresponding objective values. In addition, a probabilistic generative model is presented for estimating the noise distribution. This enables us to determine the noise distribution parametrically from a single noisy input, using the distribution of the noise-free constituent of the noisy input estimated from the training dataset as a prior. Experiments conducted using artificial and real datasets show that the proposed method suppresses the overfitting of the regression function for noisy inputs and that the root mean squared errors of the predictions are smaller compared with those of an existing method.