FedSyL: Computation-Efficient Federated Synergy Learning on Heterogeneous IoT Devices

Abstract

As a popular privacy-preserving model training technique, Federated Learning (FL) enables multiple end-devices to collaboratively train Deep Neural Network (DNN) models without exposing local privately-owned data. According to the FL paradigm, resource-constrained end-devices in IoT should perform model training which is computation-intensive, whereas the edge server occupied with powerful computation capability only performs model aggregation. Due to the above unbalanced computation pattern, IoT-oriented FL is time-consuming and inefficient. In order to alleviate the computation burden of end-devices, recent countermeasures introduce the edge server to assist end-devices in model training. However, existing works neither efficiently address the computation heterogeneity across end-devices nor reduce the leakage risk of data privacy. To this end, we propose a Federated Synergy Learning (FedSyL) paradigm which innovatively strikes a balance between training efficiency and data leakage risk. We explore the complicated relationship between the local training latency and multi-dimensional training configurations, and design a uniform training latency prediction method by applying the polynomial quadratic regression analysis. Additionally, we design the optimal model offloading strategy with the consideration of resource limitation and computation heterogeneity of end-devices, so as to accurately assign capability=matched device-side sub-models for heterogeneous end-devices. We implement FedSyL on a real test-bed comprising multiple heterogeneous end-devices. Experimental results demonstrate the superiority of FedSyL on training efficiency and privacy protection.