Exploring the Privacy-Accuracy Trade-Off Using Adaptive Gradient Clipping in Federated Learning

IF 6.7 2区计算机科学 Q1 ENGINEERING, MULTIDISCIPLINARY

IEEE Transactions on Network Science and Engineering Pub Date : 2025-02-28 DOI:10.1109/TNSE.2025.3546777

Benteng Zhang;Yingchi Mao;Xiaoming He;Ping Ping;Huawei Huang;Jie Wu

{"title":"Exploring the Privacy-Accuracy Trade-Off Using Adaptive Gradient Clipping in Federated Learning","authors":"Benteng Zhang;Yingchi Mao;Xiaoming He;Ping Ping;Huawei Huang;Jie Wu","doi":"10.1109/TNSE.2025.3546777","DOIUrl":null,"url":null,"abstract":"In Differentially Private Federated Learning (DP-FL), gradient clipping can prevent excessive noise from being added to the gradient and ensure that the impact of noise is within a controllable range. However, state-of-the-art methods adopt fixed or imprecise clipping thresholds for gradient clipping, which is not adaptive to the changes in the gradients. This issue can lead to a significant degradation in accuracy while training the global model. To this end, we propose Differential Privacy Federated Adaptive gradient Clipping based on gradient Norm (DP-FedACN). DP-FedACN can calculate the decay rate of the clipping threshold by considering the overall changing trend of the gradient norm. Furthermore, DP-FedACN can accurately adjust the clipping threshold for each training round according to the actual changes in gradient norm, clipping loss, and decay rate. Experimental results demonstrate that DP-FedACN can maintain privacy protection performance similar to that of DP-FedAvg under member inference attacks and model inversion attacks. DP-FedACN significantly outperforms DP-FedAGNC and DP-FedDDC in privacy protection metrics. Additionally, the test accuracy of DP-FedACN is approximately 2.61%, 1.01%, and 1.03% higher than the other three baseline methods, respectively. DP-FedACN can improve the global model training accuracy while ensuring the privacy protection of the model. All experimental results demonstrate that the proposed DP-FedACN can help find a fine-grained privacy-accuracy trade-off in DP-FL.","PeriodicalId":54229,"journal":{"name":"IEEE Transactions on Network Science and Engineering","volume":"12 3","pages":"2254-2265"},"PeriodicalIF":6.7000,"publicationDate":"2025-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network Science and Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10908083/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

In Differentially Private Federated Learning (DP-FL), gradient clipping can prevent excessive noise from being added to the gradient and ensure that the impact of noise is within a controllable range. However, state-of-the-art methods adopt fixed or imprecise clipping thresholds for gradient clipping, which is not adaptive to the changes in the gradients. This issue can lead to a significant degradation in accuracy while training the global model. To this end, we propose Differential Privacy Federated Adaptive gradient Clipping based on gradient Norm (DP-FedACN). DP-FedACN can calculate the decay rate of the clipping threshold by considering the overall changing trend of the gradient norm. Furthermore, DP-FedACN can accurately adjust the clipping threshold for each training round according to the actual changes in gradient norm, clipping loss, and decay rate. Experimental results demonstrate that DP-FedACN can maintain privacy protection performance similar to that of DP-FedAvg under member inference attacks and model inversion attacks. DP-FedACN significantly outperforms DP-FedAGNC and DP-FedDDC in privacy protection metrics. Additionally, the test accuracy of DP-FedACN is approximately 2.61%, 1.01%, and 1.03% higher than the other three baseline methods, respectively. DP-FedACN can improve the global model training accuracy while ensuring the privacy protection of the model. All experimental results demonstrate that the proposed DP-FedACN can help find a fine-grained privacy-accuracy trade-off in DP-FL.

查看原文本刊更多论文

利用自适应梯度裁剪探索联邦学习中隐私与准确性的权衡

在差分私有联邦学习（DP-FL）中，梯度裁剪可以防止过多的噪声加入到梯度中，保证噪声的影响在可控范围内。然而，现有的梯度裁剪方法采用固定或不精确的裁剪阈值，不能适应梯度的变化。在训练全局模型时，这个问题会导致精度的显著下降。为此，我们提出了基于梯度范数的差分隐私联邦自适应梯度裁剪（DP-FedACN）。DP-FedACN可以通过考虑梯度范数的整体变化趋势来计算裁剪阈值的衰减率。此外，DP-FedACN可以根据梯度范数、裁剪损失和衰减率的实际变化，准确地调整每个训练轮的裁剪阈值。实验结果表明，在成员推理攻击和模型反转攻击下，DP-FedACN仍能保持与dp - fedag相似的隐私保护性能。dp - fedcn在隐私保护指标上显著优于dp - fedcn和dp - feddc。此外，DP-FedACN的检测准确率比其他三种基线方法分别高出约2.61%、1.01%和1.03%。DP-FedACN可以在保证模型隐私保护的同时，提高模型的全局训练精度。所有实验结果表明，所提出的DP-FedACN可以帮助在DP-FL中找到细粒度的隐私-准确性权衡。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Network Science and Engineering Engineering-Control and Systems Engineering

CiteScore

12.60

自引率

9.10%

发文量

393

期刊介绍： The proposed journal, called the IEEE Transactions on Network Science and Engineering (TNSE), is committed to timely publishing of peer-reviewed technical articles that deal with the theory and applications of network science and the interconnections among the elements in a system that form a network. In particular, the IEEE Transactions on Network Science and Engineering publishes articles on understanding, prediction, and control of structures and behaviors of networks at the fundamental level. The types of networks covered include physical or engineered networks, information networks, biological networks, semantic networks, economic networks, social networks, and ecological networks. Aimed at discovering common principles that govern network structures, network functionalities and behaviors of networks, the journal seeks articles on understanding, prediction, and control of structures and behaviors of networks. Another trans-disciplinary focus of the IEEE Transactions on Network Science and Engineering is the interactions between and co-evolution of different genres of networks.