Large language models and unsupervised feature learning: implications for log analysis

IF 1.8 4区计算机科学 Q3 TELECOMMUNICATIONS

Annals of Telecommunications Pub Date : 2024-04-04 DOI:10.1007/s12243-024-01028-2

引用次数: 0

Abstract

Log file analysis is increasingly being addressed through the use of large language models (LLM). LLM provides the mechanism for discovering embeddings for distinguishing between different behaviors present in log files. In this work, we are interested in discriminating between normal and anomalous behaviors via an unsupervised learning approach. To this end, firstly five recent LLM architectures are evaluated over six different log files. Then, further research is conducted to explicitly quantify the significance of performing self-supervised fine-tuning on the LLMs. Moreover, we show that the quality of an (unsupervised) feature map used to make the overall (normal/anomalous) predictions may also benefit from an AutoEncoder stage between LLM and feature map. Such an AutoEncoder provides significant reductions in the cost of training the feature map and typically improves the quality of the resulting predictions.

查看原文本刊更多论文

大型语言模型和无监督特征学习：对日志分析的影响

摘要日志文件分析越来越多地通过使用大型语言模型（LLM）来解决。LLM 提供了发现嵌入的机制，以区分日志文件中存在的不同行为。在这项工作中，我们感兴趣的是通过无监督学习方法来区分正常行为和异常行为。为此，我们首先通过六种不同的日志文件对五种最新的 LLM 架构进行了评估。然后，我们进行了进一步的研究，以明确量化对 LLM 执行自监督微调的意义。此外，我们还表明，用于进行整体（正常/异常）预测的（无监督）特征图的质量也可能受益于 LLM 和特征图之间的自动编码器阶段。这样的自动编码器可以显著降低训练特征图的成本，通常还能提高预测结果的质量。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Annals of Telecommunications 工程技术-电信学

CiteScore

5.20

自引率

5.30%

发文量

审稿时长

4.5 months

期刊介绍： Annals of Telecommunications is an international journal publishing original peer-reviewed papers in the field of telecommunications. It covers all the essential branches of modern telecommunications, ranging from digital communications to communication networks and the internet, to software, protocols and services, uses and economics. This large spectrum of topics accounts for the rapid convergence through telecommunications of the underlying technologies in computers, communications, content management towards the emergence of the information and knowledge society. As a consequence, the Journal provides a medium for exchanging research results and technological achievements accomplished by the European and international scientific community from academia and industry.