LogFlow:大型系统的简化日志分析

Marc Platini, Thomas Ropars, Ben Pelletier, N. D. Palma
{"title":"LogFlow:大型系统的简化日志分析","authors":"Marc Platini, Thomas Ropars, Ben Pelletier, N. D. Palma","doi":"10.1145/3427796.3427808","DOIUrl":null,"url":null,"abstract":"Distributed infrastructures generate huge amount of logs that can provide useful information about the state of system, but that can be challenging to analyze. The paper presents LogFlow, a tool to help human operators in the analysis of logs by automatically constructing graphs of correlations between log entries. The core of LogFlow is an interpretable predictive model based on a Recurrent Neural Network augmented with a state-of-the-art attention layer from which correlations between log entries are deduced. To be able to deal with huge amount of data, LogFlow also relies on a new log parser algorithm that can be orders of magnitude faster than best existing log parsers. Experiments run with several system logs generated by Supercomputers and Cloud systems show that LogFlow is able to achieve more than 96% of accuracy in most cases.","PeriodicalId":335477,"journal":{"name":"Proceedings of the 22nd International Conference on Distributed Computing and Networking","volume":"82 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2020-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"3","resultStr":"{\"title\":\"LogFlow: Simplified Log Analysis for Large Scale Systems\",\"authors\":\"Marc Platini, Thomas Ropars, Ben Pelletier, N. D. Palma\",\"doi\":\"10.1145/3427796.3427808\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Distributed infrastructures generate huge amount of logs that can provide useful information about the state of system, but that can be challenging to analyze. The paper presents LogFlow, a tool to help human operators in the analysis of logs by automatically constructing graphs of correlations between log entries. The core of LogFlow is an interpretable predictive model based on a Recurrent Neural Network augmented with a state-of-the-art attention layer from which correlations between log entries are deduced. To be able to deal with huge amount of data, LogFlow also relies on a new log parser algorithm that can be orders of magnitude faster than best existing log parsers. Experiments run with several system logs generated by Supercomputers and Cloud systems show that LogFlow is able to achieve more than 96% of accuracy in most cases.\",\"PeriodicalId\":335477,\"journal\":{\"name\":\"Proceedings of the 22nd International Conference on Distributed Computing and Networking\",\"volume\":\"82 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2020-12-25\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"3\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 22nd International Conference on Distributed Computing and Networking\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3427796.3427808\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 22nd International Conference on Distributed Computing and Networking","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3427796.3427808","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 3

摘要

分布式基础设施生成大量日志,这些日志可以提供有关系统状态的有用信息,但分析这些信息可能具有挑战性。本文介绍了LogFlow,一个通过自动构建日志条目之间的关联图来帮助人类操作员分析日志的工具。LogFlow的核心是一个可解释的预测模型,该模型基于循环神经网络,并增强了最先进的关注层,从中推导出日志条目之间的相关性。为了能够处理大量数据,LogFlow还依赖于一种新的日志解析器算法,该算法可以比现有最好的日志解析器快几个数量级。在超级计算机和云系统生成的多个系统日志上运行的实验表明,在大多数情况下,LogFlow能够达到96%以上的准确率。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
LogFlow: Simplified Log Analysis for Large Scale Systems
Distributed infrastructures generate huge amount of logs that can provide useful information about the state of system, but that can be challenging to analyze. The paper presents LogFlow, a tool to help human operators in the analysis of logs by automatically constructing graphs of correlations between log entries. The core of LogFlow is an interpretable predictive model based on a Recurrent Neural Network augmented with a state-of-the-art attention layer from which correlations between log entries are deduced. To be able to deal with huge amount of data, LogFlow also relies on a new log parser algorithm that can be orders of magnitude faster than best existing log parsers. Experiments run with several system logs generated by Supercomputers and Cloud systems show that LogFlow is able to achieve more than 96% of accuracy in most cases.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信