Federated learning for heterogeneous neural networks with layer similarity relations in Cloud–Edge–End scenarios

Abstract

Federated Learning (FL) aims to allow numerous clients to participate in collaborative training in an efficient communication manner without exchanging private data. Traditional FL assumes that all clients have sufficient local resources to train models with the same architecture, and does not consider the reality that clients may struggle to deploy the same model across devices with varying computational resources. To address this, we propose a heterogeneous FL method, HNN-LSFL, in which the edge server first aggregates the clients of the homogeneous model, and then the cloud server selectively aligns and aggregates the knowledge between the heterogeneous models according to the layer similarity. This Cloud–Edge–End tiered architecture effectively utilizes the powerful computing power of cloud servers, reduces the computational cost of multiple alignment and aggregation of heterogeneous models, and reduces the communication cost with the cloud, which is more suitable for large-scale client scenarios. By identifying layer similarities, the method finds commonalities between different models, enabling more valuable aggregations and reducing the transmission of unnecessary parameters. We also evaluated HNN-LSFL on heterogeneous datasets, demonstrating that it not only improves the utilization of local client resources but also optimizes FL performance. By transmitting fewer model parameters, it reduces the risk of privacy leaks and proves to be superior in FL tasks with heterogeneous models compared to current state-of-the-art heterogeneous FL algorithms.