Arena: Adaptive real-time update anomaly prediction in cloud systems

Abstract

In current cloud systems, their monitoring relies strongly on rule-based and supervised-learning-based detection methods for anomaly detection. These methods require either some knowledge provided by an expert system or monitoring data to be labeled as a training set. In practice, the systems behavior changes over time. It is difficult to adjust the rules or re-train detection model for these methods. In this paper, we present an Adaptive REal-time update uNsupervised Anomaly prediction system (Arena) for cloud systems. Arena uses a clustering technique based on a density spatial clustering algorithm to identify clusters and outliers. We propose two prediction strategies to improve the ability to predict anomaly and a real-time update strategy by adding new monitoring points into Arenas model. To improve the prediction efficiency and reduce the scale of the model, we adopt a pruning method to remove redundant points. The anomaly data used in the experiments was collected from the Yahoo Lab and the component based system of enterprise T. The experimental results show that our proposed methods can achieve high prediction accuracy compared to existing methods. Realtime update strategy can improve the prediction performance. The pruning method can further reduce the scale of the model and demonstrates the prediction efficiency.