Constructive neural network ensemble for regression tasks in high dimensional spaces

Abstract

This research focuses on the development of constructive neural networks (NN)for regression tasks in high dimensional spaces. A constructive algorithm which is referred to as modified cascade correlation (MCC) has been developed. MCC has several improvements relative to the original algorithm. They include stopping the training when the minimum squared error on a small unseen dataset is reached. This method is known to improve the generalization ability of the NN, i.e. its ability to accurately predict cases not in the training set. The subject of this paper is to investigate committee networks trained with the MCC. A mathematical function is used to study the generalization properties of the network for input space dimension ranging from five to thirty. The study shows that "ensemble averaged" network committees greatly improve the generalization performance of the MCC algorithm. Areas of further research are outlined and include investigating other types of committees.