Effect of decay functions on the generalization ability of TWDRLS algorithms

Abstract

Artificial neural networks trained with a regularization term in the energy function have been shown to perform well in improving the generalization ability and reducing the complexity of the network. In a previous study, we proposed a new version of the TWDRLS algorithm with a generalized regularizer in the energy function to make it suitable for target learning. In this paper, we introduce three new decay functions to study the effect of the shape and intensity of the decay functions on the generalization ability of the trained network. Computer simulations show that the regularizer with a weak decaying effect for small weights but a relatively strong decaying effect for large ones makes the networks exhibit a better generalization ability.