Privacy-Preserving Frank-Wolfe on Shuffle Model

IF 0.9 4区 数学 Q3 MATHEMATICS, APPLIED
Ling-jie Zhang, Shi-song Wu, Hai Zhang
{"title":"Privacy-Preserving Frank-Wolfe on Shuffle Model","authors":"Ling-jie Zhang,&nbsp;Shi-song Wu,&nbsp;Hai Zhang","doi":"10.1007/s10255-024-1095-6","DOIUrl":null,"url":null,"abstract":"<div><p>In this paper, we design the differentially private variants of the classical Frank-Wolfe algorithm with shuffle model in the optimization of machine learning. Under weak assumptions and the generalized linear loss (GLL) structure, we propose a noisy Frank-Wolfe with shuffle model algorithm (NoisyFWS) and a noisy variance-reduced Frank-Wolfe with the shuffle model algorithm (NoisyVRFWS) by adding calibrated laplace noise under shuffling scheme in the <i>ℓ</i><sub><i>p</i></sub>(<i>p</i> ∈ [1, 2])-case, and study their privacy as well as utility guarantees for the Hölder smoothness GLL. In particular, the privacy guarantees are mainly achieved by using advanced composition and privacy amplification by shuffling. The utility bounds of the NoisyFWS and NoisyVRFWS are analyzed and obtained the optimal excess population risks <span>\\({\\cal O}({n^{ - {{1 + \\alpha } \\over {4\\alpha }}}} + {{\\log (d)\\sqrt {\\log ({1 \\mathord{\\left/ {\\vphantom {1 \\delta }} \\right.} \\delta })} } \\over {n\\epsilon\\,}})\\)</span> and <span>\\({\\cal O}({n^{ - {{1 + \\alpha } \\over {4\\alpha }}}} + {{\\log (d)\\sqrt {\\log ({1 \\mathord{\\left/ {\\vphantom {1 \\delta }} \\right.} \\delta })} } \\over {{n^2\\epsilon}\\,}})\\)</span> with gradient complexity <span>\\({\\cal O}({n^{ - {{{{(1 + \\alpha )}^2}} \\over {4{\\alpha ^2}}}}})\\)</span> for <span>\\(\\alpha \\in \\left[ {{1 \\mathord{\\left/ {\\vphantom {1 {\\sqrt 3 ,\\,1}}} \\right.} {\\sqrt 3 ,\\,1}}} \\right]\\)</span>. It turns out that the risk rates under shuffling scheme are a nearly-dimension independent rate, which is consistent with the previous work in some cases. In addition, there is a vital tradeoff between (<i>α, L</i>)-Hölder smoothness GLL and the gradient complexity. The linear gradient complexity <span>\\({\\cal O}(n)\\)</span> is showed by the parameter <i>α</i> = 1.</p></div>","PeriodicalId":6951,"journal":{"name":"Acta Mathematicae Applicatae Sinica, English Series","volume":"40 4","pages":"887 - 907"},"PeriodicalIF":0.9000,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Acta Mathematicae Applicatae Sinica, English Series","FirstCategoryId":"100","ListUrlMain":"https://link.springer.com/article/10.1007/s10255-024-1095-6","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 0

Abstract

In this paper, we design the differentially private variants of the classical Frank-Wolfe algorithm with shuffle model in the optimization of machine learning. Under weak assumptions and the generalized linear loss (GLL) structure, we propose a noisy Frank-Wolfe with shuffle model algorithm (NoisyFWS) and a noisy variance-reduced Frank-Wolfe with the shuffle model algorithm (NoisyVRFWS) by adding calibrated laplace noise under shuffling scheme in the p(p ∈ [1, 2])-case, and study their privacy as well as utility guarantees for the Hölder smoothness GLL. In particular, the privacy guarantees are mainly achieved by using advanced composition and privacy amplification by shuffling. The utility bounds of the NoisyFWS and NoisyVRFWS are analyzed and obtained the optimal excess population risks \({\cal O}({n^{ - {{1 + \alpha } \over {4\alpha }}}} + {{\log (d)\sqrt {\log ({1 \mathord{\left/ {\vphantom {1 \delta }} \right.} \delta })} } \over {n\epsilon\,}})\) and \({\cal O}({n^{ - {{1 + \alpha } \over {4\alpha }}}} + {{\log (d)\sqrt {\log ({1 \mathord{\left/ {\vphantom {1 \delta }} \right.} \delta })} } \over {{n^2\epsilon}\,}})\) with gradient complexity \({\cal O}({n^{ - {{{{(1 + \alpha )}^2}} \over {4{\alpha ^2}}}}})\) for \(\alpha \in \left[ {{1 \mathord{\left/ {\vphantom {1 {\sqrt 3 ,\,1}}} \right.} {\sqrt 3 ,\,1}}} \right]\). It turns out that the risk rates under shuffling scheme are a nearly-dimension independent rate, which is consistent with the previous work in some cases. In addition, there is a vital tradeoff between (α, L)-Hölder smoothness GLL and the gradient complexity. The linear gradient complexity \({\cal O}(n)\) is showed by the parameter α = 1.

洗牌模型上的隐私保护弗兰克-沃尔夫
本文设计了机器学习优化中带有洗牌模型的经典弗兰克-沃尔夫算法的不同私有变体。在弱假设和广义线性损失(GLL)结构下,我们提出了带洗牌模型的噪声弗兰克-沃尔夫算法(NoisyFWS)和在ℓp(p∈ [1, 2])情况下,通过在洗牌方案下添加校准的拉普拉斯噪声而实现的带洗牌模型的噪声方差降低弗兰克-沃尔夫算法(NoisyVRFWS),并研究了它们的隐私性以及霍尔德平滑性 GLL 的效用保证。其中,隐私保证主要是通过使用高级合成和洗牌的隐私放大来实现的。分析了NoisyFWS和NoisyVRFWS的效用边界,并得到了最优过剩人口风险({\cal O}({n^{ - {{1 + \alpha }\over {4\alpha }}}}+ {{log (d)\sqrt {log ({1 \mathord\{left/ {\vphantom {1 \delta }} \right.} \delta })} }}\over {n\epsilon\,}}) and\({\cal O}({n^{ - {{1 +\alpha })\over {4\alpha }}}}+ {{log (d)sqrt {log ({1 \mathord\{left/ {\vphantom {1 \delta }} \right.} \delta })} }}\over {{n^2\epsilon}\})\) with gradient complexity \({\cal O}({n^{ - {{{{(1 +\alpha )}^2}}\over {4{alpha ^2}}}}})}) for \(\alpha \in \left[ {{1 \mathord{ \left/ { \vphantom {1 { \sqrt 3 ,\,1}}})\right.}{sqrt 3 ,\,1}}\right]\).结果表明,洗牌方案下的风险率是一个几乎与维度无关的比率,这在某些情况下与之前的工作是一致的。此外,(α, L)-荷尔德平滑度 GLL 与梯度复杂度之间存在着重要的权衡。线性梯度复杂度 \({c\al O}(n)\) 由参数 α = 1 表示。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
1.30
自引率
0.00%
发文量
70
审稿时长
3.0 months
期刊介绍: Acta Mathematicae Applicatae Sinica (English Series) is a quarterly journal established by the Chinese Mathematical Society. The journal publishes high quality research papers from all branches of applied mathematics, and particularly welcomes those from partial differential equations, computational mathematics, applied probability, mathematical finance, statistics, dynamical systems, optimization and management science.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信