Stochastic backpropagation: a learning algorithm for generalization problems

Abstract

Neural networks have traditionally been applied to recognition problems, and most learning algorithms are tailored to those problems. The authors discuss the requirements of learning for generalization, which is NP-complete and cannot be approached by traditional methods based on gradient descent. They present a stochastic learning algorithm based on simulated annealing in weight space. The convergence properties and feasibility of the algorithm are verified.<>