Communication Efficient Federated Learning Framework with Local Momentum

With the recent progress of AI, large amount of data generated in distributed Internet of Things (IoT) devices can be used to build different kinds of models that are helpful to improve people’s daily life. For example, language models can improve the speech recognition performance. Federated learning enables the distributed clients to jointly learn a model with data preserve in local, which provide a promising solution to leverage the massive data. However, in federated learning, the model learned by the local devices need to be repeatedly transmit to the server, which poses communication overhead. To tackle the communication issue, this paper proposes a communication efficient federated learning framework that utilize local momentum term to accelerate the convergence speed. Convergence guarantee under non-convex case is provided. Experiment on EMNIST and CIFAR10 dataset demonstrate that proposed method can effectively increase the convergence speed.