Nonparametric multiple regression by projection on non-compactly supported bases

Abstract

We study the nonparametric regression estimation problem with a random design in \({\mathbb{R}}^{p}\) with \(p\ge 2\). We do so by using a projection estimator obtained by least squares minimization. Our contribution is to consider non-compact estimation domains in \({\mathbb {R}}^{p}\), on which we recover the function, and to provide a theoretical study of the risk of the estimator relative to a norm weighted by the distribution of the design. We propose a model selection procedure in which the model collection is random and takes into account the discrepancy between the empirical norm and the norm associated with the distribution of design. We prove that the resulting estimator automatically optimizes the bias-variance trade-off in both norms, and we illustrate the numerical performance of our procedure on simulated data.