Some Implications of System Dynamics Analysis of Discrete-Time Recurrent Neural Networks for Learning Algorithms Design

Abstract

It is not clear so far what the implications of bifurcations in Discrete-Time Recurrent Neural Networks dynamics are with respect to learning algorithms. Previous studies discussed different phenomena in a general purpose framework, and here we are going to discuss in more detail. We perform an analysis of the dynamics of a neuron with feedback in order to find the different behaviors that it shows depending on the magnitude of the offset weight, the input weight and the feedback weight. We calculate the bifurcation manifolds that show the regions where the neuron behavior changes. We discuss the implications that these findings can have for the design of DTRNN learning algorithms.