Back-propagation as the solution of differential-algebraic equations for artificial neural network training

Abstract

The backpropagation algorithm for neural network training is formulated as the solution of a set of sparse differential algebraic equations (DAE). These equations are then solved as a function of time. The solution of the differential equations is performed using an implicit integrator with adjustable time step. The topology of the Jacobian matrix associated with the DAE's is illustrated. A training example is included.<>