Analyzing Vector Orthogonalization Algorithms

SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 2, Page 829-846, June 2024.
Abstract. Computer implementations of vector orthogonalization algorithms produce a sequence of supposedly orthogonal vectors, but rounding-errors can cause loss of orthogonality and rank. Nevertheless these computational algorithms can be very effective as parts of various methods. We develop a general theory based on the augmented orthogonal matrix developed in [SIAM J. Matrix Anal. Appl., 31 (2009), pp. 565–583] that can be applied to any such algorithm. This can be combined with a rounding-error analysis of the algorithm to analyze its finite-precision behavior. We apply this combination to prove that a particular Lanczos tridiagonalization of a Hermitian matrix always computes components for which backward-stable solutions to [math], [math], exist. If an appropriate rounding-error analysis is available, the approach can apparently be applied to any computation producing a sequence of supposedly orthogonal [math]-vectors, where a linear combination of these vectors is intended to approximate some quantity.