Decentralized non-convex online optimization with adaptive momentum estimation and quantized communication

Abstract

In this work, we consider the decentralized non-convex online optimization problem over an undirected network. To solve the problem over a communication-efficient manner, we propose a novel quantized decentralized adaptive momentum gradient descent algorithm based on the adaptive momentum estimation methods, where quantified information is exchanged between agents. The proposed algorithm not only can effectively reduce the data transmission volume but also contribute to improved convergence. Theoretical analysis proves that the proposed algorithm can achieve sublinear dynamic regret under appropriate step-size and quantization level, which matches the convergence of the decentralized online algorithm with exact-communication. Extensive simulations are given to demonstrate the efficacy of the algorithm.