Randomized Quantization for Privacy in Resource Constrained Machine Learning at-the-Edge and Federated Learning

Abstract

The increasing adoption of machine learning at the edge (ML-at-the-edge) and federated learning (FL) presents a dual challenge: ensuring data privacy as well as addressing resource constraints such as limited computational power, memory, and communication bandwidth. Traditional approaches typically apply differentially private stochastic gradient descent (DP-SGD) to preserve privacy, followed by quantization techniques as a post-processing step to reduce model size and communication overhead. However, this sequential framework introduces inherent drawbacks, as quantization alone lacks privacy guarantees and often introduces errors that degrade model performance. In this work, we propose randomized quantization as an integrated solution to address these dual challenges by embedding randomness directly into the quantization process. This approach enhances privacy while simultaneously reducing communication and computational overhead. To achieve this, we introduce Randomized Quantizer Projection Stochastic Gradient Descent (RQP-SGD), a method designed for ML-at-the-edge that embeds DP-SGD within a randomized quantization-based projection during model training. For federated learning, we develop Gaussian Sampling Quantization (GSQ), which integrates discrete Gaussian sampling into the quantization process to ensure local differential privacy (LDP). Unlike conventional methods that rely on Gaussian noise addition, GSQ achieves privacy through discrete Gaussian sampling while improving communication efficiency and model utility across distributed systems. Through rigorous theoretical analysis and extensive experiments on benchmark datasets, we demonstrate that these methods significantly enhance the utility-privacy trade-off and computational efficiency in both ML-at-the-edge and FL systems. RQP-SGD is evaluated on MNIST and the Breast Cancer Diagnostic dataset, showing an average 10.62% utility improvement over the deterministic quantization-based projected DP-SGD while maintaining (1.0, 0)-DP. In federated learning tasks, GSQ-FL improves accuracy by an average 11.52% over DP-FedPAQ across MNIST and FashionMNIST under non-IID conditions. Additionally, GSQ-FL outperforms DP-FedPAQ by 16.54% on CIFAR-10 and 8.7% on FEMNIST.