Automated Feature Selection for Anomaly Detection in Network Traffic Data

Makiya Nakashima, A. Sim, Youngsoo Kim, Jong-Hoi Kim, Jinoh Kim
{"title":"Automated Feature Selection for Anomaly Detection in Network Traffic Data","authors":"Makiya Nakashima, A. Sim, Youngsoo Kim, Jong-Hoi Kim, Jinoh Kim","doi":"10.1145/3446636","DOIUrl":null,"url":null,"abstract":"\n Variable selection (also known as\n feature selection\n ) is essential to optimize the learning complexity by prioritizing features, particularly for a massive, high-dimensional dataset like network traffic data. In reality, however, it is not an easy task to effectively perform the feature selection despite the availability of the existing selection techniques. From our initial experiments, we observed that the existing selection techniques produce different sets of features even under the same condition (e.g., a static size for the resulted set). In addition, individual selection techniques perform inconsistently, sometimes showing better performance but sometimes worse than others, thereby simply relying on one of them would be risky for building models using the selected features. More critically, it is demanding to automate the selection process, since it requires laborious efforts with intensive analysis by a group of experts otherwise. In this article, we explore challenges in the automated feature selection with the application of network anomaly detection. We first present our ensemble approach that benefits from the existing feature selection techniques by incorporating them, and one of the proposed ensemble techniques based on greedy search works highly consistently showing comparable results to the existing techniques. We also address the problem of when to stop to finalize the feature elimination process and present a set of methods designed to determine the number of features for the reduced feature set. Our experimental results conducted with two recent network datasets show that the identified feature sets by the presented ensemble and stopping methods consistently yield comparable performance with a smaller number of features to conventional selection techniques.\n","PeriodicalId":178565,"journal":{"name":"ACM Trans. Manag. Inf. Syst.","volume":"62 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"10","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACM Trans. Manag. Inf. Syst.","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3446636","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 10

Abstract

Variable selection (also known as feature selection ) is essential to optimize the learning complexity by prioritizing features, particularly for a massive, high-dimensional dataset like network traffic data. In reality, however, it is not an easy task to effectively perform the feature selection despite the availability of the existing selection techniques. From our initial experiments, we observed that the existing selection techniques produce different sets of features even under the same condition (e.g., a static size for the resulted set). In addition, individual selection techniques perform inconsistently, sometimes showing better performance but sometimes worse than others, thereby simply relying on one of them would be risky for building models using the selected features. More critically, it is demanding to automate the selection process, since it requires laborious efforts with intensive analysis by a group of experts otherwise. In this article, we explore challenges in the automated feature selection with the application of network anomaly detection. We first present our ensemble approach that benefits from the existing feature selection techniques by incorporating them, and one of the proposed ensemble techniques based on greedy search works highly consistently showing comparable results to the existing techniques. We also address the problem of when to stop to finalize the feature elimination process and present a set of methods designed to determine the number of features for the reduced feature set. Our experimental results conducted with two recent network datasets show that the identified feature sets by the presented ensemble and stopping methods consistently yield comparable performance with a smaller number of features to conventional selection techniques.
网络流量数据异常检测的自动特征选择
变量选择(也称为特征选择)是通过对特征进行优先排序来优化学习复杂性的关键,特别是对于像网络流量数据这样的大规模高维数据集。然而,在现实中,尽管现有的选择技术可用,但要有效地执行特征选择并不是一件容易的事情。从我们最初的实验中,我们观察到,即使在相同的条件下(例如,结果集的静态大小),现有的选择技术也会产生不同的特征集。此外,单个选择技术的执行不一致,有时表现出更好的性能,但有时比其他技术差,因此仅仅依赖其中一种技术对于使用所选特征构建模型是有风险的。更关键的是,它要求自动化选择过程,因为它需要一组专家进行艰苦的努力和深入的分析。在本文中,我们探讨了网络异常检测在自动特征选择中的应用所面临的挑战。我们首先提出了我们的集成方法,该方法通过合并现有的特征选择技术而受益,并且提出的基于贪婪搜索的集成技术之一工作高度一致,显示出与现有技术相当的结果。我们还解决了何时停止以完成特征消除过程的问题,并提出了一组用于确定减少特征集的特征数量的方法。我们用两个最近的网络数据集进行的实验结果表明,通过所提出的集成和停止方法识别的特征集与传统的选择技术相比,在较少数量的特征上始终产生相当的性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信