A biologically plausible neural network training algorithm with composite chaos

Abstract

Chaos appears in many real and artificial systems. Inspired from the presence of chaos in human brain, we attempt to formulate neural network (NN) training method. The method uses a composite chaotic learning rate (CCLR) to train a neural network. CCLR generates a composite chaotic time series consisting of three different chaotic sources such as Mackey Glass, Logistic Map and Lorenz Attractor and a rescaled version of the series is used as learning rate (LR) during NN training. It gives two advantages — similarity with biological phenomena and possibility of jumping from local minima. In addition, the weight update may be accelerated in the local minimum zone due to chaotic variation of LR. CCLR is extensively tested on five real world benchmark classification problems such as diabetes, time series, horse, glass and soybean. The proposed CCLR outperforms the existing BP and BPCL in terms of generalization ability and also convergence rate.