Anomaly Transformer Ensemble Model for Cloud Data Anomaly Detection

Abstract

The stability and user trust in cloud services depends on prompt detection and response to diverse anomalies. This study focuses on an Ensemble-based anomaly detection methodology that integrates log data with computing resource metrics, aiming to overcome the limitations of traditional single-data models. To process the unstructured nature of log data, we use the Drain Parser to transform it into a structured format, and Doc2Vec embeds it. The study adheres to a reconstruction-based approach for anomaly detection, specifically building upon the Anomaly Transformer model. The proposed model leverages the concept of an Anomaly Transformer based on the Attention mechanism. It integrates preprocessed metric data with log data for effective anomaly detection. Experiments were conducted using metric and log data collected from real-world cloud environments. The model’s performance was evaluated based on accuracy, recall, precision, f1 score, and AUROC. The results demonstrate that our proposed Ensemble-based model outperforms traditional models such as LSTM, VAR, and deeplog.