Research and Implementation of Asynchronous Transmission and Update Strategy for Federated Learning

Abstract

In today's era, there is a growing concern about data privacy and security. Data is the foundation of machine learning, and due to the complexity of the reality, data often exists in silos. The great development of computing and storage capacity of edge devices has made it possible to train machine learning models on edge devices with data. Federated learning is a special kind of distributed machine learning in which multiple distributed clients use local data to train a model and upload the resulting model parameters to a central server for aggregation and iteration to finally obtain a global model with good generalization performance. The interaction between the clients and the server is model parameters rather than the data itself, which effectively protects data privacy and security. After research, it is found that Federated Learning is mainly divided into two aggregation models: synchronous aggregation and asynchronous aggregation. In this paper, the classical FedAvg synchronous federated average aggregation algorithm is analyzed. The central server needs to wait for all participating nodes to be trained before aggregating and updating the model, which is inconvenient to achieve global synchronization on resource heterogeneous clients. To improve the utilization of computational resources, FedAsync, an asynchronous federated learning aggregation mechanism with a receivable window based on dual weights, which are data volume size weight and staleness weight, is proposed. And FedAvg and FedAsync are tested and analyzed. Based on this, the impact on federated learning due to asynchronous data arrival rate and limited communication resources is further explored.