Least Squares and the Not-Normal Equations

SIAM Review, Volume 67, Issue 4, Page 865-872, December 2025.
Abstract.For many of the classic problems of linear algebra, effective and efficient numerical algorithms exist, particularly for situations where dimensions are not too large. The linear least squares problem is one such example: excellent algorithms exist when [math] factorization is feasible. However, for large-dimensional (often sparse) linear least squares problems there currently exist good solution algorithms only for well-conditioned problems or for problems where there are lots of data but only a few variables in the solution. Such approaches ubiquitously employ normal equations and so have to contend with conditioning issues. We explore some alternative approaches that we characterize as not-normal equations where conditioning may not be such an issue.