A fast progressive local learning regression ensemble of generalized regression neural networks

Abstract

The Generalized Regression Neural Network (GRNN) is a memory-based supervised learning neural network that performs non linear regressions and output estimation. However, if the number of the hidden layer neurons grows large, the evaluation of an unknown sample has a substantial computational cost. Whereas the GRNN run time can be reduced by parallelism, the computational load can be decreased by neuron reduction that compresses pattern neurons into fewer kernels. While such global models have been studied for a long time, there is another solution; that of local learning algorithms which use neighbourhoods to learn local parameters and create on the fly a local model specifically designed for any particular testing point. For this purpose we create a Progressive Local Learning Ensemble of many local GRNN models. Optimizing the number of k nearest neighbor neurons the method reduces substantially the cost of training as well as of predicting an unknown sample.