Multi-granularity Weighted Federated Learning in Heterogeneous Mobile Edge Computing Systems

Abstract

As a promising framework for distributed learning in mobile edge computing scenarios, federated learning (FL) allows multiple mobile devices to train a model collaboratively without transferring raw data and exposing user privacy. However, vanilla FL schemes are still facing to problems in edge computing, where the diversity of tasks and devices causes the non-IID and multi-granularity data with model heterogeneity. It becomes a pressing challenge to jointly training edge devices accompanied by these problems, while vanilla FL only discusses them separately. To this end, we consider tailoring FL to adapt to mobile edge environments, which focus on solving the problems of collaborative training of edge devices with multi-granularity heterogeneous models under different data distributions. In particular, we proposed a distance-based FL for the same type of edge devices that provides personalized models to avoid the negative impact of non-IID data on model aggregation. Further, we design a bi-directional guidance method with a prior attention mechanism, which can transfer knowledge among edge devices with multi-granulairty and multi-scale models. The experimental results show that our proposed mechanisms significantly improve training performance compared to other baselines on IID and non-IID data. Furthermore, the bi-directional guidance significantly improves convergence efficiency and accuracy performance for finer and coarser granularity edge devices, respectively.