HT-FL: Hybrid Training Federated Learning for Heterogeneous Edge-Based IoT Networks

Abstract

With the continuous rolling-out of edge computing, Federated Learning (FL) has become a promising solution for intelligent Internet-of-things (IoT). In addition to resource constraints, deploying FL schemes in IoT networks is greatly challenged by heterogeneity in multiple dimensions. While heterogeneity in data distribution and computation capability has been extensively studied, the impact of distinct, even hybrid training paradigms on FL performances remains largely unknown. To answer this open question in the IoT context, we propose a Hybrid-Training Federated Learning (HT-FL) algorithm for the power-constrained IoT networks, incorporating both sequential and parallel training that naturally adapts to various sub-network topologies, while greatly reducing the energy consumption during the training stage. We demonstrate through analysis that the convergence of HT-FL is theoretically guaranteed, achieving $O (\frac{1}{\sqrt{K}})$ for carefully chosen learning rates. Experiments on multiple datasets show that, the proposed HT-FL outperforms existing FL schemes on multiple training tasks under various data distribution settings, while reducing an average of 20% energy consumption. In a more practical sense, a self-adaptive parameter-tuning strategy is also designed for HT-FL deployment, which can be easily extended to other multi-layer FL schemes in complex application scenarios.