Dynamic Adaptive Federated Learning on Local Long-Tailed Data

Abstract

Federated learning provides privacy protection to the collaborative training of global model based on distributed private data. The local private data is often in the presence of long-tailed distribution in reality, which downgrades the performance and causes biased results. In this paper, we propose a dynamic adaptive federated learning optimization algorithm with the Grey Wolf Optimizer and Markov Chain, named FedWolf, to solve the problems of performance degradation and result bias caused by the local long-tailed data. FedWolf is launched with a set of randomly initialized parameters instead of a shared parameter employed by existing methods. Then multi-level participants are elected based on the F1 scores calculated from the uploaded parameters. A dynamic weighting strategy based on the participant level is used to adaptively update parameters without artificial control. The above parameter updating is modelled as a Markov Process. After all communication rounds are completed, the future performance (including the probability of each participant is elected as different participant level) of participants is predicted through the historical Markov states. Finally, the probability of each participant is elected as the level 1 is used as the contribution weight and the global model is obtained through dynamic contribution weight aggregating. We introduce the Gini index to evaluate the bias of classification results. Extensive experiments are conducted to validate the effectiveness of FedWolf in solving the problems of performance cracks and categorization result bias as well as the robustness of adaptive parameter updating in resisting outliers and malicious users.