Embedding discriminant directions in backpropagation

Abstract

A two-phase backpropagation algorithm is presented. In the first phase the directions of the weight vectors of the first hidden layer are constrained to remain in directions suitably chosen by pattern recognition, data compression, or speech and image processing techniques. Then, the constraints are removed and the standard backpropagation algorithm takes over to further minimize the error function. The first phase swiftly situates the weight vectors in a good position which can serve as the initialization of the standard backpropagation algorithm. The generality of its application, its simplicity, and the shorter training time it requires, makes this approach attractive.<>