The Direct Solution of Weighted and Equality Constrained Least-Squares Problems

Abstract

We consider methods to solve two closely related linear least-squares problems. The first problem is that of minimizing ${\|f - Ex\|}_2 $ subject to the constraint $Cx = g$. We call this the linear least-squares (LSE) problem. The second is that of minimizing \[ \left\| {\left( {\begin{array}{*{20}c} {\tau g} \\ f \\ \end{array} } \right) - \left( {\begin{array}{*{20}c} {\tau C} \\ E \\ \end{array} } \right)x} \right\|_2 \] for some large weight $\tau $. This second problem is called the WLS problem.A column-pivoting strategy based entirely upon the constraint matrix C is developed for solving the weighted least-squares (WLS) problem. This strategy allows the user to perform the factorization of $(\begin{array}{*{20}c} {\tau C} \\ E \\ \end{array} )$ in stable fashion while needing to access no more than one row of E at a time. Moreover, if the matrix E is changed without changing the sparsity pattern or the matrix C, then the pivoting need not be redone. We can simply reuse the same column ordering. This kind of computation frequently arises in optimization contexts.An error analysis of the method is presented. It is shown to be closely related to the error analysis of a procedure attributed to Bjorck and Golub in their solving of the LSE problem. The sparsity properties of the algorithm are demonstrated on some Harwell test matrices.