Training algorithm based on Newton's method with dynamic error control

The use of Newton's method with dynamic error control as a training algorithm for the backpropagation (BP) neural network is considered. Theoretically, it can be proved that Newton's method is convergent in the second-order while the most widely used steepest-descent method is convergent in the first-order. This suggests that Newton's method might be a faster algorithm for the BP network. The updating equations of the two methods are analyzed in detail to extract some important properties with reference to the error surface characteristics. The common benchmark XOR problem is used to compare the performance of the methods.<>