Investigation on Different Kernel Functions for Weighted Kernel Regression in Solving Small Sample Problems

Abstract

Previously, weighted kernel regression (WKR) has proved to solve small problems. The existing WKR has been successfully solved rational functions with very few samples. The design and development of WKR is important in order to extend the capability of the technique with various kernel functions. Based on WKR, a simple iteration technique is employed to estimate the weight parameters with Gaussian as a kernel function before WKR can be used in predicting the unseen test samples. In this paper, however, we investigate various kernel functions with Particle Swarm Optimization (PSO) as weight estimators as it offers such flexibility in defining the objective function. Hence, PSO has the capability to solve non-closed form solution problem as we also introduce regularization term with L1 norm in defining the objective function as to solve training sample, which corrupted by noise. Through a number of computational experiments, the investigation results show that the prediction quality of WKR is primarily dominated by the smoothing parameter selection rather than the type of kernel function.