Improved CBP learning with output bias decomposition

Abstract

Choosing a network size is a difficult problem in neural network modelling. In many recent studies constructive or destructive methods that add or delete connections, neurons, layers have been studied for solving this problem In this work we consider the constructive approach. In particular we address the construction of feedforward networks by the use of improved constructive backpropagation that utilizes output bias decomposition scheme. The proposed improved scheme is shown to be beneficial especially in regression type problems like time series modelling. Namely, our time series prediction experiments demonstrate that the improved method is competitive in terms of modelling performance and training time compared to the well known cascade-correlation method.