Approximate Solutions of Linear Systems at a Universal Rate

Let be invertible, unknown, and given. We are interested in approximate solutions: vectors such that is small. We prove that for all , there is a composition of orthogonal projections onto the hyperplanes generated by the rows of , where , which maps the origin to a vector satisfying . We note that this upper bound on is independent of the matrix . This procedure is stable in the sense that . The existence proof is based on a probabilistically refined analysis of the randomized Kaczmarz method, which seems to achieve this rate when solving for with high likelihood. We also prove a general version for matrices with and full rank.