Methodology of neural network compression for multi-sensor transducer network models based on edge computing principles

Ivan Lobachev, S. Antoshchuk, Mykola A. Hodovychenko
{"title":"Methodology of neural network compression for multi-sensor transducer network models based on edge computing principles","authors":"Ivan Lobachev, S. Antoshchuk, Mykola A. Hodovychenko","doi":"10.15276/hait.03.2021.3","DOIUrl":null,"url":null,"abstract":"This paper focuses on the development of a methodology to compress neural networks thatis based on the mechanism of prun-ingthe hidden layer neurons. The aforementioned neural networks are created in order to process the data generated by numerous sensors present in a transducer network that would be employed in a smart building. The proposed methodology implements a single approach for the compression of both Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) that are used for the tasks of classification and regression. The main principle behind this method is based on the dropout mechanism, which is employed as a regulation mechanism for the neural networks. The idea behind the method proposed consists of selecting optimal exclusion probability of a hidden layer neuron, based on the redundancy of the said neuron. The novelty of this method is theusage of a custom compression network thatis based on an RNN, which allows us to determine the redundancy parameter not just in a sin-gle hidden layer, but across severallayers. The additional novelty aspect consists of an iterative optimization of the network-optimizer, to have continuous improvement of the redundancy parameter calculator of the input network. For the experimental evalu-ation of the proposed methodology, the task of image recognition with a low-resolution camera was chosen, the CIFAR10 dataset was used to emulate the scenario. The VGGNet Convolutional Neural Network, that contains convolutional and fully connected lay-ers, was used as the network under test for the purposes of this experiment. The following two methods were taken as the analogous state of the art, the MagBase method, which is based on the sparcification principle as well as the method which is based on rarefied representation by employing the approach of rarefied encoding SFAC. The results of the experiment demonstrated that the amount of parameters in the compressed model is only 2.56% of the original input model. This has allowed us to reduce the logical output time by 93.7% and energy consumption by 94.8%. The proposed method allows to effectively usingdeep neural networks in transducer networks that utilize the architecture of edge computing. This in turn allows the system to process the data in real time, reduce the energy consumption and logical output time as well as lower the memory and storage requirements of real-world applications.","PeriodicalId":375628,"journal":{"name":"Herald of Advanced Information Technology","volume":"10 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Herald of Advanced Information Technology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.15276/hait.03.2021.3","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

This paper focuses on the development of a methodology to compress neural networks thatis based on the mechanism of prun-ingthe hidden layer neurons. The aforementioned neural networks are created in order to process the data generated by numerous sensors present in a transducer network that would be employed in a smart building. The proposed methodology implements a single approach for the compression of both Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) that are used for the tasks of classification and regression. The main principle behind this method is based on the dropout mechanism, which is employed as a regulation mechanism for the neural networks. The idea behind the method proposed consists of selecting optimal exclusion probability of a hidden layer neuron, based on the redundancy of the said neuron. The novelty of this method is theusage of a custom compression network thatis based on an RNN, which allows us to determine the redundancy parameter not just in a sin-gle hidden layer, but across severallayers. The additional novelty aspect consists of an iterative optimization of the network-optimizer, to have continuous improvement of the redundancy parameter calculator of the input network. For the experimental evalu-ation of the proposed methodology, the task of image recognition with a low-resolution camera was chosen, the CIFAR10 dataset was used to emulate the scenario. The VGGNet Convolutional Neural Network, that contains convolutional and fully connected lay-ers, was used as the network under test for the purposes of this experiment. The following two methods were taken as the analogous state of the art, the MagBase method, which is based on the sparcification principle as well as the method which is based on rarefied representation by employing the approach of rarefied encoding SFAC. The results of the experiment demonstrated that the amount of parameters in the compressed model is only 2.56% of the original input model. This has allowed us to reduce the logical output time by 93.7% and energy consumption by 94.8%. The proposed method allows to effectively usingdeep neural networks in transducer networks that utilize the architecture of edge computing. This in turn allows the system to process the data in real time, reduce the energy consumption and logical output time as well as lower the memory and storage requirements of real-world applications.
基于边缘计算原理的多传感器传感器网络模型神经网络压缩方法
本文研究了一种基于隐层神经元剪枝机制的神经网络压缩方法。上述神经网络的创建是为了处理由传感器网络中存在的众多传感器产生的数据,这些传感器将用于智能建筑。提出的方法实现了一种单一的方法来压缩卷积神经网络(CNN)和循环神经网络(RNN),用于分类和回归任务。该方法的主要原理是基于dropout机制,该机制被用作神经网络的调节机制。所提出的方法背后的思想包括根据所述神经元的冗余度选择隐藏层神经元的最优排除概率。这种方法的新颖之处在于使用了基于RNN的自定义压缩网络,这使得我们不仅可以在单个隐藏层中确定冗余参数,还可以跨多个层确定冗余参数。另一个新颖性方面包括对网络优化器的迭代优化,以不断改进输入网络的冗余参数计算器。为了对所提出的方法进行实验评估,选择了低分辨率相机的图像识别任务,并使用CIFAR10数据集对该场景进行了模拟。本实验使用包含卷积层和全连接层的VGGNet卷积神经网络作为待测网络。本文将基于规范原则的MagBase方法和采用稀疏编码SFAC方法的基于稀疏表示的MagBase方法作为目前的同类方法。实验结果表明,压缩模型中的参数数量仅为原始输入模型的2.56%。这使我们能够将逻辑输出时间减少93.7%,能耗减少94.8%。该方法允许在利用边缘计算架构的传感器网络中有效地使用深度神经网络。这反过来又使系统能够实时处理数据,减少能耗和逻辑输出时间,并降低实际应用程序的内存和存储要求。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信