Stochastic Fixed-Point Iterations for Nonexpansive Maps: Convergence and Error Bounds

SIAM Journal on Control and Optimization, Volume 62, Issue 1, Page 191-219, February 2024.
Abstract. We study a stochastically perturbed version of the well-known Krasnoselskii–Mann iteration for computing fixed points of nonexpansive maps in finite dimensional normed spaces. We discuss sufficient conditions on the stochastic noise and stepsizes that guarantee almost sure convergence of the iterates towards a fixed point and derive nonasymptotic error bounds and convergence rates for the fixed-point residuals. Our main results concern the case of a martingale difference noise with variances that can possibly grow unbounded. This supports an application to reinforcement learning for average reward Markov decision processes, for which we establish convergence and asymptotic rates. We also analyze in depth the case where the noise has uniformly bounded variance, obtaining error bounds with explicit computable constants.