Deterministic neuron: a model for faster learning

Abstract

Training in most neural network architectures are currently being done by updating the weights of the network in a way to reduce some error measures. The well-known backpropagation algorithm and some other training algorithms use this approach. Obviously, this has been very successful in mimicking the way the biological neurons do their function. But the problem of slow learning and getting trapped in local minimas of error function domain deserve serious investigation. Various models are proposed with various levels of success to get rid of these two problems. In this work, we propose a deterministic model of the neuron, that guarantees faster learning by modifying the nonlinearity associated with each neuron. Only one such neuron is required to solve the generalized N-bit parity problem.<>