Analysis of weight decay regularisation in NNARX nonlinear identification

Abstract

This paper presents the analysis of weight decay regularisation, which is one of artificial neural network generalisation categories, in modelling nonlinear behaviour of steam temperature in distillation essential oil extraction system. The modelling is based on the neural network autoregressive with exogenous input structure. During the network training, the optimisation of the network weights has been carried out by minimisation the error through the Levenberg-Marquardt algorithm (LMA). In the weight decay regularisation network training, the LMA has been modified. Several results on unregularised and regularised trainings have been presented, compared and concluded. The results showed that the optimal weights are obtained with the moderate regularisation of the network training.