An RBM Anomaly Detector for the Cloud

Abstract

Failures are unavoidable in complex software systems, and the intrinsic characteristics of cloud systems amplify the problem. Predicting failures before their occurrence by detecting anomalies in system metrics is a viable solution to enable failure preventing or mitigating actions. The most promising approaches for predicting failures exploit statistical analysis or machine learning to reveal anomalies and their correlation with possible failures. Statistical analysis approaches result in far too many false positives, which severely hinder their practical applicability, while accurate machine learning approaches need extensive training with seeded faults, which is often impossible in operative cloud systems. In this paper, we propose EmBeD, Energy-Based anomaly Detection in the cloud, an approach to detect anomalies at runtime based on the free energy of a Restricted Boltzmann Machine (RBM) model. The free energy is a stochastic function that can be used to efficiently score anomalies for detecting outliers. EmBeD analyzes the system behavior from raw metric data, does not require extensive training with seeded faults, and classifies the relation of anomalous behaviors with future failures with very few false positives. The experimental results presented in this paper confirm that EmBeD can precisely predict failure-prone behavior without training with seeded faults, thus overcoming the main limitations of current approaches.