Penalty OBS scheme for feedforward neural network

Abstract

This paper presented a new scheme called penalty OBS (optimal brain surgeon) for the feedforward neural network learning. The penalty OBS scheme takes OBS pruning case as a penalty item of the network cost function, and develops two applied methods based on the common algorithms of network learning. As a novel revision of OBS, the new scheme not only saves the runtime to calculate Hessian matrix after training, but also improves the generalization a lot and keeps over-linear rapidity of convergence. The simulating results verified the advantages