FAccT 2022 : 2022 5th ACM Conference on Fairness, Accountability, and Transparency : June 21-24, 2022, Seoul, South Korea. ACM Conference on Fairness, Accountability, and Transparency (5th : 2022 : Seoul, Korea; Online)最新文献

筛选

英文中文

Dynamic Privacy Budget Allocation Improves Data Efficiency of Differentially Private Gradient Descent. 动态隐私预算分配提高了差分隐私梯度下降法的数据效率

FAccT 2022 : 2022 5th ACM Conference on Fairness, Accountability, and Transparency : June 21-24, 2022, Seoul, South Korea. ACM Conference on Fairness, Accountability, and Transparency (5th : 2022 : Seoul, Korea; Online) Pub Date : 2022-06-01 Epub Date: 2022-06-20 DOI: 10.1145/3531146.3533070

Junyuan Hong, Zhangyang Wang, Jiayu Zhou

{"title":"Dynamic Privacy Budget Allocation Improves Data Efficiency of Differentially Private Gradient Descent.","authors":"Junyuan Hong, Zhangyang Wang, Jiayu Zhou","doi":"10.1145/3531146.3533070","DOIUrl":"10.1145/3531146.3533070","url":null,"abstract":"Protecting privacy in learning while maintaining the model performance has become increasingly critical in many applications that involve sensitive data. Private Gradient Descent (PGD) is a commonly used private learning framework, which noises gradients based on the Differential Privacy protocol. Recent studies show that dynamic privacy schedules of decreasing noise magnitudes can improve loss at the final iteration, and yet theoretical understandings of the effectiveness of such schedules and their connections to optimization algorithms remain limited. In this paper, we provide comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions. We first present a dynamic noise schedule minimizing the utility upper bound of PGD, and show how the noise influence from each optimization step collectively impacts utility of the final model. Our study also reveals how impacts from dynamic noise influence change when momentum is used. We empirically show the connection exists for general non-convex losses, and the influence is greatly impacted by the loss curvature.","PeriodicalId":73013,"journal":{"name":"FAccT 2022 : 2022 5th ACM Conference on Fairness, Accountability, and Transparency : June 21-24, 2022, Seoul, South Korea. ACM Conference on Fairness, Accountability, and Transparency (5th : 2022 : Seoul, Korea; Online)","volume":"2022 ","pages":"11-35"},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10115558/pdf/nihms-1888054.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9791014","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0