Lattice point sets for efficient kernel smoothing models

Abstract

This work addresses the problem of learning an unknown function from data when local models are employed. In particular, kernel smoothing models are considered, which use kernels in a straightforward fashion by modeling the output as a weighted average of values observed in a neighborhood of the input. Such models are a popular alternative to other kernel paradigms, such as support vector machines (SVM), due to their very light computational burden. The purpose of this work is to prove that a smart deterministic selection of the observation points can be advantageous with respect to input data coming from a pure random sampling. Apart from the theoretical interest, this has a practical implication in all the cases in which one can control the generation of the input samples (e.g., in applications from robotics, dynamic programming, optimization, mechanics, etc.) To this purpose, lattice point sets (LPSs), a special kind of sampling schemes commonly employed for efficient numerical integration, are investigated. It is proved that building local kernel smoothers using LPSs guarantees universal approximation property with better rates with respect to i.i.d. sampling. Then, a rule for automatic kernel width selection, making the computational burden of building the model negligible, is introduced to show how the regular structure of the lattice can lead to practical advantages. Simulation results are also provided to test in practice the performance of the proposed methods.