用于保护隐私的神经网络的高效同态 Argmax 近似算法

IF 1.8 Q3 COMPUTER SCIENCE, INFORMATION SYSTEMS
Peng Zhang, Ao Duan, Hengrui Lu
{"title":"用于保护隐私的神经网络的高效同态 Argmax 近似算法","authors":"Peng Zhang, Ao Duan, Hengrui Lu","doi":"10.3390/cryptography8020018","DOIUrl":null,"url":null,"abstract":"Privacy-preserving neural networks offer a promising solution to train and predict without user privacy leakage, and fully homomorphic encryption (FHE) stands out as one of the key technologies, as it enables homomorphic operations over encrypted data. However, only addition and multiplication homomorphisms are supported by FHE, and thus, it faces huge challenges when implementing non-linear functions with ciphertext inputs. Among the non-linear functions in neural networks, one may refer to the activation function, the argmax function, and maximum pooling. Inspired by using a composition of low-degree minimax polynomials to approximate sign and argmax functions, this study focused on optimizing the homomorphic argmax approximation, where argmax is a mathematical operation that identifies the index of the maximum value within a given set of values. For the method that uses compositions of low-degree minimax polynomials to approximate argmax, in order to further reduce approximation errors and improve computational efficiency, we propose an improved homomorphic argmax approximation algorithm that includes rotation accumulation, tree-structured comparison, normalization, and finalization phases. And then, the proposed homomorphic argmax algorithm was integrated into a neural network structure. Comparative experiments indicate that the network with our proposed argmax algorithm achieved a slight increase in accuracy while significantly reducing the inference latency by 58%, as the homomorphic sign and rotation operations were rapidly reduced.","PeriodicalId":36072,"journal":{"name":"Cryptography","volume":null,"pages":null},"PeriodicalIF":1.8000,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"An Efficient Homomorphic Argmax Approximation for Privacy-Preserving Neural Networks\",\"authors\":\"Peng Zhang, Ao Duan, Hengrui Lu\",\"doi\":\"10.3390/cryptography8020018\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Privacy-preserving neural networks offer a promising solution to train and predict without user privacy leakage, and fully homomorphic encryption (FHE) stands out as one of the key technologies, as it enables homomorphic operations over encrypted data. However, only addition and multiplication homomorphisms are supported by FHE, and thus, it faces huge challenges when implementing non-linear functions with ciphertext inputs. Among the non-linear functions in neural networks, one may refer to the activation function, the argmax function, and maximum pooling. Inspired by using a composition of low-degree minimax polynomials to approximate sign and argmax functions, this study focused on optimizing the homomorphic argmax approximation, where argmax is a mathematical operation that identifies the index of the maximum value within a given set of values. For the method that uses compositions of low-degree minimax polynomials to approximate argmax, in order to further reduce approximation errors and improve computational efficiency, we propose an improved homomorphic argmax approximation algorithm that includes rotation accumulation, tree-structured comparison, normalization, and finalization phases. And then, the proposed homomorphic argmax algorithm was integrated into a neural network structure. Comparative experiments indicate that the network with our proposed argmax algorithm achieved a slight increase in accuracy while significantly reducing the inference latency by 58%, as the homomorphic sign and rotation operations were rapidly reduced.\",\"PeriodicalId\":36072,\"journal\":{\"name\":\"Cryptography\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.8000,\"publicationDate\":\"2024-05-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Cryptography\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.3390/cryptography8020018\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cryptography","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3390/cryptography8020018","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

隐私保护神经网络为在不泄露用户隐私的情况下进行训练和预测提供了一种前景广阔的解决方案,而全同态加密(FHE)是其中一项关键技术,因为它可以对加密数据进行同态运算。然而,全同态加密只支持加法和乘法同态,因此在使用密码文本输入实现非线性函数时面临巨大挑战。神经网络中的非线性函数包括激活函数、argmax 函数和最大池化函数。受使用低度最小多项式的组合来近似符号和 argmax 函数的启发,本研究重点关注同态 argmax 近似的优化,其中 argmax 是一种数学运算,用于确定给定值集合中最大值的索引。对于使用低度最小多项式组成来近似 argmax 的方法,为了进一步减少近似误差和提高计算效率,我们提出了一种改进的同态 argmax 近似算法,包括旋转累积、树状结构比较、归一化和最终确定阶段。然后,将所提出的同态 argmax 算法集成到神经网络结构中。对比实验表明,由于同态符号和旋转操作迅速减少,采用我们提出的 argmax 算法的网络在推理延迟大幅减少 58% 的同时,准确率也略有提高。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
An Efficient Homomorphic Argmax Approximation for Privacy-Preserving Neural Networks
Privacy-preserving neural networks offer a promising solution to train and predict without user privacy leakage, and fully homomorphic encryption (FHE) stands out as one of the key technologies, as it enables homomorphic operations over encrypted data. However, only addition and multiplication homomorphisms are supported by FHE, and thus, it faces huge challenges when implementing non-linear functions with ciphertext inputs. Among the non-linear functions in neural networks, one may refer to the activation function, the argmax function, and maximum pooling. Inspired by using a composition of low-degree minimax polynomials to approximate sign and argmax functions, this study focused on optimizing the homomorphic argmax approximation, where argmax is a mathematical operation that identifies the index of the maximum value within a given set of values. For the method that uses compositions of low-degree minimax polynomials to approximate argmax, in order to further reduce approximation errors and improve computational efficiency, we propose an improved homomorphic argmax approximation algorithm that includes rotation accumulation, tree-structured comparison, normalization, and finalization phases. And then, the proposed homomorphic argmax algorithm was integrated into a neural network structure. Comparative experiments indicate that the network with our proposed argmax algorithm achieved a slight increase in accuracy while significantly reducing the inference latency by 58%, as the homomorphic sign and rotation operations were rapidly reduced.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Cryptography
Cryptography Mathematics-Applied Mathematics
CiteScore
3.80
自引率
6.20%
发文量
53
审稿时长
11 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信