Federated Learning with Anomaly Client Detection and Decentralized Parameter Aggregation

Abstract

Federated learning is a framework for machine learning that is dedicated to data privacy protection. In federated learning, system cannot fully control the behavior of clients which can be faulty. These behaviors include sharing arbitrary faulty gradients and delaying the process of sharing due to Byzantine attacks or clients’ own software and hardware failures. In federated learning, the parameter server may also be faulty during gradient collection and aggregation, mainly including gradient-based training data inference and model parameter faulty update. The above problems may lead to reduced accuracy of federated learning model training, leakage of client privacy, etc. Existing research enhances the robustness of federated learning by exploiting the decentralization and immutability of Blockchain. For untrusted clients, most research is based on Byzantine fault tolerance to defend against clients indiscriminately, and may cause model accuracy reduction. In addition, most of the research focus on unencrypted gradients, and there is insufficient research on dealing with client anomalies in the case of gradient encryption. For untrusted parameter servers, existing research has problems in energy overhead and scalability. Aiming at the problems above, this paper studies the robustness of federated learning, and proposes a blockchain-based federated learning parameter update architecture PUS-FL. Through experiments simulating distributed machine learning on neural networks, we demonstrate that the anomaly detection algorithm of PUS-FL outperforms conventional gradient filters including geometric median, Multi-Krum and trimmed mean. In addition, our experiments also verify that the scalability-enhanced parameter aggregation consensus algorithm proposed in this paper(SE-PBFT) improves consensus scalability by reducing communication complexity.