利用低函数张量分解和深度解卷进行网络流量自适应异常检测

Lukas Schynol, Marius Pesavento
{"title":"利用低函数张量分解和深度解卷进行网络流量自适应异常检测","authors":"Lukas Schynol, Marius Pesavento","doi":"arxiv-2409.11529","DOIUrl":null,"url":null,"abstract":"Anomaly detection (AD) is increasingly recognized as a key component for\nensuring the resilience of future communication systems. While deep learning\nhas shown state-of-the-art AD performance, its application in critical systems\nis hindered by concerns regarding training data efficiency, domain adaptation\nand interpretability. This work considers AD in network flows using incomplete\nmeasurements, leveraging a robust tensor decomposition approach and deep\nunrolling techniques to address these challenges. We first propose a novel\nblock-successive convex approximation algorithm based on a regularized\nmodel-fitting objective where the normal flows are modeled as low-rank tensors\nand anomalies as sparse. An augmentation of the objective is introduced to\ndecrease the computational cost. We apply deep unrolling to derive a novel deep\nnetwork architecture based on our proposed algorithm, treating the\nregularization parameters as learnable weights. Inspired by Bayesian\napproaches, we extend the model architecture to perform online adaptation to\nper-flow and per-time-step statistics, improving AD performance while\nmaintaining a low parameter count and preserving the problem's permutation\nequivariances. To optimize the deep network weights for detection performance,\nwe employ a homotopy optimization approach based on an efficient approximation\nof the area under the receiver operating characteristic curve. Extensive\nexperiments on synthetic and real-world data demonstrate that our proposed deep\nnetwork architecture exhibits a high training data efficiency, outperforms\nreference methods, and adapts seamlessly to varying network topologies.","PeriodicalId":501034,"journal":{"name":"arXiv - EE - Signal Processing","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Adaptive Anomaly Detection in Network Flows with Low-Rank Tensor Decompositions and Deep Unrolling\",\"authors\":\"Lukas Schynol, Marius Pesavento\",\"doi\":\"arxiv-2409.11529\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Anomaly detection (AD) is increasingly recognized as a key component for\\nensuring the resilience of future communication systems. While deep learning\\nhas shown state-of-the-art AD performance, its application in critical systems\\nis hindered by concerns regarding training data efficiency, domain adaptation\\nand interpretability. This work considers AD in network flows using incomplete\\nmeasurements, leveraging a robust tensor decomposition approach and deep\\nunrolling techniques to address these challenges. We first propose a novel\\nblock-successive convex approximation algorithm based on a regularized\\nmodel-fitting objective where the normal flows are modeled as low-rank tensors\\nand anomalies as sparse. An augmentation of the objective is introduced to\\ndecrease the computational cost. We apply deep unrolling to derive a novel deep\\nnetwork architecture based on our proposed algorithm, treating the\\nregularization parameters as learnable weights. Inspired by Bayesian\\napproaches, we extend the model architecture to perform online adaptation to\\nper-flow and per-time-step statistics, improving AD performance while\\nmaintaining a low parameter count and preserving the problem's permutation\\nequivariances. To optimize the deep network weights for detection performance,\\nwe employ a homotopy optimization approach based on an efficient approximation\\nof the area under the receiver operating characteristic curve. Extensive\\nexperiments on synthetic and real-world data demonstrate that our proposed deep\\nnetwork architecture exhibits a high training data efficiency, outperforms\\nreference methods, and adapts seamlessly to varying network topologies.\",\"PeriodicalId\":501034,\"journal\":{\"name\":\"arXiv - EE - Signal Processing\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - EE - Signal Processing\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.11529\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - EE - Signal Processing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.11529","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

异常检测(AD)越来越被认为是确保未来通信系统弹性的关键组成部分。虽然深度学习已经显示出最先进的异常检测性能,但其在关键系统中的应用却受到训练数据效率、领域适应性和可解释性等问题的阻碍。本研究利用不完整的测量数据考虑网络流中的反向增量,并利用稳健的张量分解方法和深度滚动技术来应对这些挑战。我们首先提出了一种基于正则化模型拟合目标的新型块继承凸近似算法,其中正常流量被建模为低秩张量,异常流量被建模为稀疏。为了降低计算成本,我们引入了一个增强目标。我们基于所提出的算法,应用深度开卷法推导出一种新型的深度网络架构,并将其标准化参数视为可学习的权重。在贝叶斯方法的启发下,我们扩展了模型架构,以对每流和每时间步统计进行在线适应,从而提高了 AD 性能,同时保持了较低的参数数量,并保留了问题的包换方差。为了优化深度网络权重以提高检测性能,我们采用了一种同调优化方法,该方法基于对接收器工作特征曲线下面积的有效近似。在合成数据和真实世界数据上进行的广泛实验表明,我们提出的深度网络架构具有很高的训练数据效率,优于参考方法,并能无缝适应不同的网络拓扑结构。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Adaptive Anomaly Detection in Network Flows with Low-Rank Tensor Decompositions and Deep Unrolling
Anomaly detection (AD) is increasingly recognized as a key component for ensuring the resilience of future communication systems. While deep learning has shown state-of-the-art AD performance, its application in critical systems is hindered by concerns regarding training data efficiency, domain adaptation and interpretability. This work considers AD in network flows using incomplete measurements, leveraging a robust tensor decomposition approach and deep unrolling techniques to address these challenges. We first propose a novel block-successive convex approximation algorithm based on a regularized model-fitting objective where the normal flows are modeled as low-rank tensors and anomalies as sparse. An augmentation of the objective is introduced to decrease the computational cost. We apply deep unrolling to derive a novel deep network architecture based on our proposed algorithm, treating the regularization parameters as learnable weights. Inspired by Bayesian approaches, we extend the model architecture to perform online adaptation to per-flow and per-time-step statistics, improving AD performance while maintaining a low parameter count and preserving the problem's permutation equivariances. To optimize the deep network weights for detection performance, we employ a homotopy optimization approach based on an efficient approximation of the area under the receiver operating characteristic curve. Extensive experiments on synthetic and real-world data demonstrate that our proposed deep network architecture exhibits a high training data efficiency, outperforms reference methods, and adapts seamlessly to varying network topologies.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信