Algorithms for nonlinear problems which use discrete approximations to derivatives

Abstract

The most desirable algorithms for nonlinear programming problems call for obtaining the gradient of the objective and the Jacobian of the constraint function. The analytic form is often impossible and almost always impractical to obtain. The usual expedient is to use difference quotients to approximate the partial derivatives. This paper is concerned with the theoretical and practical ramifications of such modifications to basic algorithms. Among the methods surveyed are steepest descent, Stewart's modification of the Davidon-Fletcher-Powell method, the Levenberg-Marquardt method, Newton's method, and the nonlinear reduced gradient method. Numerical results are included in the presentation.