Privacy-Preserving SGD on Shuffle Model

Abstract

In this paper, we consider an exceptional study of differentially private stochastic gradient descent (SGD) algorithms in the stochastic convex optimization (SCO). The majority of the existing literature requires that the losses have additional assumptions, such as the loss functions with Lipschitz, smooth and strongly convex, and uniformly bounded of the model parameters, or focus on the Euclidean (i.e. ${\mathcal{l}}_2^d$ ) setting. However, these restrictive requirements exclude many popular losses, including the absolute loss and the hinge loss. By loosening the restrictions, we proposed two differentially private SGD without shuffle model and with shuffle model algorithms (in short, DP-SGD-NOS and DP-SGD-S) for the $\left(\alpha ,L\right)$ -Hölder smooth loss by adding calibrated Laplace noise under no shuffling scheme and shuffling scheme in the ${\mathcal{l}}_p^d$ -setting for $p\in \left[\mathrm{1,2}\right]$ . We provide privacy guarantees by using advanced composition and privacy amplification techniques. We also analyze the convergence bounds of the DP-SGD-NOS and DP-SGD-S and obtain the optimal excess population risks $\mathcal{O}\left(1/\sqrt{n}+\sqrt{d\log \left(1/\delta \right)}/nϵ\right)$ and $\mathcal{O}\left(1/\sqrt{n}+\sqrt{d\log \left(1/\delta \right)\log \left(n/\delta \right)}/n^{\left(4+\alpha \right)/\left(2\left(1+\alpha \right)\right)}ϵ\right)$ up to logarithmic factors with gradient complexity

分享

查看原文 本刊更多论文

Shuffle模型上的隐私保护SGD

在本文中，我们考虑了随机凸优化(SCO)中微分私有随机梯度下降(SGD)算法的特殊研究。现有文献大多要求损失具有附加假设，如损失函数具有Lipschitz、光滑且强凸、模型参数均匀有界等;或专注于欧几里得(即二维)设置。然而，这些限制性要求排除了许多常见的损耗，包括绝对损耗和铰链损耗。通过放宽限制，我们提出了两种不同的私有SGD算法(即DP-SGD-NOS和DP-SGD-S)，用于α，L -Hölder在无洗牌方案和在L p中加入校准拉普拉斯噪声的平滑损失D -设为p∈1,2。我们通过使用先进的组合和隐私放大技术提供隐私保证。我们还分析了DP-SGD-NOS和DP-SGD-S的收敛界，得到了最优超额群体风险O 1 / n +D log1 / δ/ nO (1 / n + dlog1 / δ logN / δ 在本文中，我们考虑了随机凸优化(SCO)中微分私有随机梯度下降(SGD)算法的特殊研究。现有文献大多要求损失具有附加假设，如损失函数具有Lipschitz、光滑且强凸、模型参数均匀有界等;或专注于欧几里得(即二维)设置。然而，这些限制性要求排除了许多常见的损耗，包括绝对损耗和铰链损耗。通过放宽限制，我们提出了两种不同的私有SGD算法(即DP-SGD-NOS和DP-SGD-S)，用于α，L -Hölder在无洗牌方案和在L p中加入校准拉普拉斯噪声的平滑损失D -设为p∈1,2。我们通过使用先进的组合和隐私放大技术提供隐私保证。

本文章由计算机程序翻译，如有差异，请以英文原文为准。