Reflective block Kaczmarz algorithms for least squares

Abstract

In Steinerberger (2021) [23] and Shao (2023) [21], two new types of Kaczmarz algorithms, which share some similarities, for consistent linear systems were proposed. These two algorithms not only compete with many previous Kaczmarz algorithms but, more importantly, reveal some interesting new geometric properties of solutions to linear systems that are not obvious from the standard viewpoint of the Kaczmarz algorithm. In this paper, we comprehensively study these two algorithms. First, we theoretically analyse the algorithms for solving least squares, which is more common in practice. Second, we extend the two algorithms to block versions and provide their rigorous estimate on the convergence rates. Third, as a theoretical complement, we provide more results on properties of the convergence rate. All these results contribute to a more thorough understanding of these algorithms.