Minimizing validation error with respect to network size and number of training epochs

Abstract

A batch training algorithm for the multilayer perceptron is developed that optimizes validation error with respect to two parameters. At the end of each training epoch, the method temporarily prunes the network and calculates the validation error versus number of hidden units curve in one pass through the validation data. Since, pruning is done at each epoch, and the best networks are saved, we optimize validation error over the number of hidden units and the number of epochs simultaneously. The number of required multiplies for the algorithm has been analyzed. The method has been compared to others in simulations and has been found to work very well.