Variogram based noise variance estimation and its use in kernel based regression

Abstract

Model-free estimates of the noise variance are important for doing model selection and setting tuning parameters. In this paper a data representation is discussed which leads to such an estimator suitable for multi-dimensional input data. The visual representation, called the differogram cloud, is based on the 2-norm of the differences of the input- and output-data. A corrected way to estimate the variance of the noise on the output measurement and a (tuning) parameter free version are derived. Connections with other existing variance estimators and numerical simulations indicate convergence of the estimators. As a special case, this paper focuses on model selection and tuning parameters of least squares support vector machines [J. Suykens, et al., 2002].