Nesterov Accelerated Gradient Tracking With Adam for Distributed Online Optimization.

Abstract

This article presents an accelerated distributed optimization algorithm for online optimization problems over large-scale networks. The proposed algorithm's iteration only relies on local computation and communication. To effectively adapt to dynamic changes and achieve a fast convergence rate while maintaining good convergence performance, we design a new algorithm called NGTAdam. This algorithm combines the Nesterov acceleration technique with an adaptive moment estimation method. The convergence of NGTAdam is evaluated by evaluating its dynamic regret through the use of linear system inequality. For online convex optimization problems, we provide an upper bound on the dynamic regret of NGTAdam, which depends on the initial conditions and the time-varying nature of the optimization problem. Moreover, we show that if the time-varying part of this upper bound is sublinear with time, the dynamic regret is also sublinear. Through a variety of numerical experiments, we demonstrate that NGTAdam outperforms state-of-the-art distributed online optimization algorithms.