FedMQ+: Towards efficient heterogeneous federated learning with multi-grained quantization

Federated Learning (FL) is a distributed machine learning paradigm that enables collaborative model training while preserving client data privacy. Although this approach significantly enhances data privacy, it introduces substantial communication overhead. Quantization techniques mitigate this challenge by compressing model parameters into fewer bits. However, traditional quantization methods are primarily implemented at the client level, often overlooking the heterogeneous importance of distinct model parameters. In our previous work, FedMQ, we explored quantization mechanisms at both inter-client and intra-client levels to improve communication efficiency. Nevertheless, this approach compromises model accuracy due to the aggregation of models with limited expressive capability. Additionally, it lacks comprehensive theoretical analysis and mathematical verification. In this paper, we propose FedMQ+, an improved framework for heterogeneous federated learning with enhanced dequantization, to optimize global model performance. First, we design a precise dequantization strategy based on normal functions to accurately reconstruct full-precision weights from the given low-precision weights. Next, we conduct a rigorous theoretical analysis of the FedMQ+, establish an upper bound for its convergence, and mathematically demonstrate its $O(1/T)$ convergence rate. Finally, we perform extensive experiments across diverse datasets and models. Experimental results demonstrate that FedMQ+ achieves significant improvements in convergence speed from 3.1% to 85.8%, while maintaining comparable model accuracy and achieving superior communication efficiency compared with state-of-the-art baselines.