Multi-Layer federated learning for bit-width and data heterogeneity in cloud-edge systems

In edge intelligence scenarios, quantized models often vary in bit-width due to hardware diversity. Inconsistent model quantization and non-IID local data create dual heterogeneity, which causes significant challenges for federated training across edges. To overcome these challenges, we introduce a novel method (multi-layer Federated Learning with Bit-width Adaptivity) for dual heterogeneity, which facilitates collaborative optimization across differently quantized models. Our method establishes multiple model layers at edges, enabling alternate execution of collaborative and local updates. During collaborative updates, edges synchronize with other edges by selecting suitable models with the same bit-width from their candidate sets. To reduce communication, a Contact Map is designed at the cloud, tracking model selection and guiding efficient collaboration. We theoretically analyze the reduction of communication burden by the Contact Map. Experimental evaluations show that our method outperforms existing approaches in terms of model accuracy in dual heterogeneous settings.