The parallel alternating direction method of multipliers: Optimal step-size or preconditioning matrix

Abstract

Alternating direction method of multipliers (ADMM) can decompose a complex problem into several smaller, more manageable subproblems, which can be solved independently. This is particularly useful for large-scale problems. However, ADMM has a slow convergence rate, especially compared to other optimization methods. In this paper, an alternating direction method of multipliers (ADMM) using two different parallel techniques is studied. First, the convergence properties of ADMM are given which can be regarded as instructions on how to design the modified ADMM. Then, by introducing the optimal step-size method and the preconditioning matrix method, the convergence rate can be increased, and researchers can use ADMM or its modifications to deal with different kinds of problems. Convergence analysis and numerical examples demonstrate our results.