MOHFL: Multi-Level One-Shot Hierarchical Federated Learning With Enhanced Model Aggregation Over Non-IID Data

Abstract

Hierarchical federated learning (HFL) is a privacy-preserving distributed machine learning framework with a client-edge-cloud hierarchy, where multiple edge servers perform partial model aggregation to reduce costly communication with the cloud server. Nevertheless, most existing HFL methods require extensive iterative communication and public datasets, which not only increase communication overhead but also raise privacy and security concerns. Moreover, non-independent and identically distributed (non-IID) data among devices can significantly impact the accuracy of the global model in HFL. To address these challenges, we propose a multi-level one-shot HFL framework (MOHFL), which aims to improve the performance of the global model in a single communication round. Specifically, we employ conditional variational autoencoders (CVAEs) as local models and use the aggregated decoders to generate an IID training set for the global model, thereby mitigating the negative impact of non-IID data. We improve the performance of CVAEs under different levels of data heterogeneity through a dominant class-based data selection method. Subsequently, an edge aggregation scheme based on multi-teacher knowledge distillation and contrastive learning is proposed to aggregate the knowledge from local decoders to edge decoders. Extensive experiments on four real-world datasets demonstrate that MOHFL is very competitive against four state-of-the-art baselines under various settings.