The Rate of Convergence of Bregman Proximal Methods: Local Geometry Versus Regularity Versus Sharpness

SIAM Journal on Optimization, Volume 34, Issue 3, Page 2440-2471, September 2024.
Abstract. We examine the last-iterate convergence rate of Bregman proximal methods—from mirror descent to mirror-prox and its optimistic variants—as a function of the local geometry induced by the prox-mapping defining the method. For generality, we focus on local solutions of constrained, nonmonotone variational inequalities, and we show that the convergence rate of a given method depends sharply on its associated Legendre exponent, a notion that measures the growth rate of the underlying Bregman function (Euclidean, entropic, or other) near a solution. In particular, we show that boundary solutions exhibit a stark separation of regimes between methods with a zero and nonzero Legendre exponent: The former converge at a linear rate, while the latter converge, in general, sublinearly. This dichotomy becomes even more pronounced in linearly constrained problems where methods with entropic regularization achieve a linear convergence rate along sharp directions, compared to convergence in a finite number of steps under Euclidean regularization.