On the Performance of the Nonsynaptic Backpropagation for Training Long-term Cognitive Networks

Abstract

Long-term Cognitive Networks (LTCNs) are recurrent neural networks for modeling and simulation. Such networks can be trained in a synaptic or nonsynaptic mode according to their goal. Nonsynaptic learning refers to adjusting the transfer function parameters while preserving the weights connecting the neurons. In that regard, the Nonsynaptic Backpropagation (NSBP) algorithm has proven successful in training LTCN based models. Despite NSBP's success, a question worthy of investigation is whether the backpropagation process is necessary when training these recurrent neural networks. This paper investigates this issue and presents three nonsynaptic learning methods that modify the original algorithm. In addition, we perform a sensitivity analysis of both the NSBP's hyperparameters and the LTCNs' learnable parameters. The main conclusions of our study are i) the backward process attached to the NSBP algorithm is not necessary to train these recurrent neural systems, and ii) there is a nonsynaptic learnable parameter that does not contribute significantly to the LTCNs' performance.