Perseus: a simple and optimal high-order method for variational inequalities

Abstract

This paper settles an open and challenging question pertaining to the design of simple and optimal high-order methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding \(x^\star \in {\mathcal {X}}\) such that \(\langle F(x), x - x^\star \rangle \ge 0\) for all \(x \in {\mathcal {X}}\). We consider the setting in which \(F: {\mathbb {R}}^d \rightarrow {\mathbb {R}}^d\) is smooth with up to \((p-1)^{\text {th}}\)-order derivatives. For \(p = 2\), the cubic regularization of Newton’s method has been extended to VIs with a global rate of \(O(\epsilon ^{-1})\) (Nesterov in Cubic regularization of Newton’s method for convex problems with constraints, Tech. rep., Université catholique de Louvain, Center for Operations Research and Econometrics (CORE), 2006). An improved rate of \(O(\epsilon ^{-2/3}\log \log (1/\epsilon ))\) can be obtained via an alternative second-order method, but this method requires a nontrivial line-search procedure as an inner loop. Similarly, the existing high-order methods based on line-search procedures have been shown to achieve a rate of \(O(\epsilon ^{-2/(p+1)}\log \log (1/\epsilon ))\) (Bullins and Lai in SIAM J Optim 32(3):2208–2229, 2022; Jiang and Mokhtari in Generalized optimistic methods for convex–concave saddle point problems, 2022; Lin and Jordan in Math Oper Res 48(4):2353–2382, 2023). As emphasized by Nesterov (Lectures on convex optimization, vol 137, Springer, Berlin, 2018), however, such procedures do not necessarily imply the practical applicability in large-scale applications, and it is desirable to complement these results with a simple high-order VI method that retains the optimality of the more complex methods. We propose a \(p^{\text {th}}\)-order method that does not require any line search procedure and provably converges to a weak solution at a rate of \(O(\epsilon ^{-2/(p+1)})\). We prove that our \(p^{\text {th}}\)-order method is optimal in the monotone setting by establishing a lower bound of \(\Omega (\epsilon ^{-2/(p+1)})\) under a generalized linear span assumption. A restarted version of our \(p^{\text {th}}\)-order method attains a linear rate for smooth and \(p^{\text {th}}\)-order uniformly monotone VIs and another restarted version of our \(p^{\text {th}}\)-order method attains a local superlinear rate for smooth and strongly monotone VIs. Further, the similar \(p^{\text {th}}\)-order method achieves a global rate of \(O(\epsilon ^{-2/p})\) for solving smooth and nonmonotone VIs satisfying the Minty condition. Two restarted versions attain a global linear rate under additional \(p^{\text {th}}\)-order uniform Minty condition and a local superlinear rate under additional strong Minty condition.