Single-Pass Nyström Approximation in Mixed Precision

Abstract

SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 3, Page 1361-1391, September 2024.
Abstract. Low-rank matrix approximations appear in a number of scientific computing applications. We consider the Nyström method for approximating a positive semidefinite matrix [math]. In the case that [math] is very large or its entries can only be accessed once, a single-pass version may be necessary. In this work, we perform a complete rounding error analysis of the single-pass Nyström method in two precisions, where the computation of the expensive matrix product with [math] is assumed to be performed in the lower of the two precisions. Our analysis gives insight into how the sketching matrix and shift should be chosen to ensure stability, implementation aspects which have been commented on in the literature but not yet rigorously justified. We further develop a heuristic to determine how to pick the lower precision, which confirms the general intuition that the lower the desired rank of the approximation, the lower the precision we can use without detriment. We also demonstrate that our mixed precision Nyström method can be used to inexpensively construct limited memory preconditioners for the conjugate gradient method and derive a bound on the condition number of the resulting preconditioned coefficient matrix. We present numerical experiments on a set of matrices with various spectral decays and demonstrate the utility of our mixed precision approach.