REC-Fed: A Robust and Efficient Clustered Federated System for Dynamic Edge Networks

Abstract

As a promising approach, Clustered Federated Learning (CFL) enables personalized model aggregation for heterogeneous clients. However, facing dynamic and open edge networks, previous CFL rarely considers the impact of dynamic client data on clustering validity, or sensitively identifies low-quality parameters from highly heterogeneous client models. Moreover, the device heterogeneity in each cluster leads to unbalanced model transmission delay, thus reducing the system efficiency. To tackle the above issues, this paper proposes a Robust and Efficient Clustered Federated System (REC-Fed). First, a Hierarchical Attention based Robust Aggregation (HARA) method is designed to realize layer-wise model customization for clients, meanwhile keeping the clustering validity under dynamic client data distribution. In addition, the fine-grained parameter detection in HARA provides a natural advantage to detect low-quality parameters, which improves the robustness of CFL systems. Second, to realize efficient synchronous model transmission, an Adaptive Model Transmission Optimization (AMTO) is proposed to jointly optimize the model compression and bandwidth allocation for heterogenous clients. Finally, we theoretically analyze the convergence of REC-Fed and conduct experiments on several personalization tasks, which demonstrate that our REC-Fed has significant improvement on flexibility, robustness and efficiency.