The Conjugate Gradient Method with neural network control

Abstract

To address the unconstrained optimization problem, the Conjugate Gradient Method (CG) uses the sequence of iterations to approach the minimum point of aim function. Because of the effect of rounding errors, many merits of CG are no longer in existence in practical use. Hence the rate of convergence is not ideal and a practical problem confronting us is how to improve conjugate gradient iteration so as to accelerate the convergence. Common improvements include better descent directions and restart strategies on the precondition of conjugate gradients. From the angle of the search step length, another major factor that influences the rate of convergence, the author proposes the use of the neural network model to introduce ‘priori knowledge’ in CG so that it may predict the next search step length. Large quantities of experimental data prove that this method can effectively improve the rate of convergence.