Analyzing and Enhancing LDP Perturbation Mechanisms in Federated Learning

Abstract

Recently, federated learning (FL) has become a prevalent algorithm to harvest data while preserving privacy. However, private information can still be compromised by local parameters during transmissions between local parties and the central server. To address this problem, local differential privacy (LDP) has been adopted. Known as federated LDP-SGD, each local device only sends perturbed parameters to the central server. However, due to the low model efficiency caused by overwhelming LDP noise, only a relaxed LDP privacy scheme, namely Gaussian mechanism, is explored in the federated LDP-SGD literature. The objective of this paper is to enable other LDP mechanisms (e.g., Laplace, Piecewise, Square Wave and Gaussian) in federated learning by enhancing their model efficiency. We first propose an analytical framework that generalizes federated LDP-SGD and derives its model efficiency. Serving as a benchmark, this framework can compare performances of different LDP mechanisms in federated learning. Based on this framework, we identify a new perspective to generally optimize federated LDP-SGD, namely, the vectorized perturbation strategy LDPVec. By only perturbing the direction of a gradient, LDPVec better preserves the descending direction of the gradient, which consequently leads to comprehensive efficiency improvements in terms of various LDP mechanisms.