Graph-Based Joint Client Clustering and Resource Allocation for Wireless Distributed Learning: A New Hierarchical Federated Learning Framework With Non-IID Data

Abstract

Hierarchical federated learning (HFL) is a key technology enabling distributed learning with reduced communication overhead. However, practical HFL systems encounter two major challenges: limited resources and data heterogeneity. In particular, limited resources can result in intolerable system latency, while heterogeneous data across clients can significantly degrade model accuracy and convergence rates. To address these issues and fully leverage the potential of HFL, we propose a novel framework called graph-based joint client and resource orchestration. This framework addresses the challenges of practical networks through joint client clustering and resource allocation. First, we propose a learning process where edge servers employ hypernetworks to achieve edge aggregation. This method can generate personalized client models and extract data distributions without directly exposing data distributions. Then, to characterize the joint effects of limited resources and data heterogeneity, we propose a graph-based modeling method and formulate a joint optimization problem that aims to balance data distributions and minimize latency. Subsequently, we propose a graph neural network-based algorithm to tackle the formulated problem with low-complexity optimization. Numerical results demonstrate significant benefits over existing algorithms in terms of convergence latency, model accuracy, scalability, and adaptability to new distributions.