An efficient accelerated gradient method with memory applicable to composite problems

Abstract

Gradient Methods with Memory store at runtime part of the oracle information obtained at previous iterations. This model allows them to outperform classical gradient methods in many situations. Solving the inner problem associated with the model does incur an overhead but, for unconstrained problems where the objective has a Lipschitz gradient, this overhead has been shown to be minimal. In this work we propose an accelerated gradient method with memory applicable to composite problems. In our method the model overhead remains negligible even for constrained problems with non-differentiable objectives. Although the inner problem cannot be solved exactly, we propose a model and choose the starting point of the inner optimization scheme in a way that prevents the accumulation of errors as the algorithm progresses. Moreover, our method dynamically adjusts the convergence guarantees to exceed those of the Fast Gradient Method. The theoretical predictions are confirmed numerically on an image deblurring problem.