Step acceleration based training algorithm for feedforward neural networks

Abstract

This paper presents a very fast step acceleration based training algorithm (SATA) for multilayer feedforward neural network training. The most outstanding virtue of this algorithm is that it does not need to calculate the gradient of the target function. In each iteration step, the computation only concentrates on the corresponding varied part. The proposed algorithm has attributes in simplicity, flexibility and feasibility, as well as high speed of convergence. Compared with the other methods, including the conventional backpropagation (BP), conjugate gradient, and weight extrapolation based BP, many simulations confirmed the superiority of this algorithm in terms of converging speed and computation time required.