Parallel Nonlinear Optimization

This paper describes the implementation of a parallel Levenberg-Marquardt algorithm on an iPSC/2. The Levenberg-Marquardt algorithm is a standard technique for non-linear least-squares optimization. For a problem with D data points and P parameters to be estimated, each iteration requires that the objective function and its P partials be evaluated at all D data points, using the current parameter estimates. Each iteration also requires the solution of a PxP linear system to obtain the next set of parameter estimates. A simple data-parallel decomposition is used where the data is evenly distributed across the nodes to parallelize the evaluations of the objective function and its partial derivatives. The performance of the method is characterized versus the number of nodes, the number of data points, and the number of parameters in the objective function. Further enhancements are also discussed.