New primal-dual algorithm for convex-concave saddle point problems

Primal-dual algorithm (PDA) is a classic and popular scheme for convex-concave saddle point problems. It is universally acknowledged that the proximal terms in the subproblems of the primal and dual variables are crucial to the convergence theory and numerical performance of primal-dual algorithms. By taking advantage of the information from the current and previous iterative points, we employ two convex combinations on all previously generated iterative points to generate two new proximal terms for the subproblems of the primal and dual variables. It is remarkable that the weight assigned to the latest iterative point is always significant with a lower bound of 0.75 and can approach 1 regardless of whether the convex coefficient is near 0 or 1. Based on two novel proximal terms, we present a new primal-dual algorithm for convex-concave saddle point problems with bilinear coupling term and establish its global convergence and an O(1/N) ergodic convergence rate. When either the primal function or the dual function is strongly convex, we accelerate the above proposed algorithm and show that the corresponding algorithm can achieve an O(1/N2) convergence rate. Since the conditions for the stepsizes of the proposed algorithm are related directly to the spectral norm of the linear transform, which is difficult to obtain in some applications, we also introduce a linesearch strategy for the above proposed primal-dual algorithm and establish its global convergence and an O(1/N) ergodic convergence rate. Some numerical experiments are conducted on matrix game and LASSO problems by comparing with other state-of-the-art algorithms, which demonstrate the effectiveness of the proposed three primal-dual algorithms.