ADE: An ensemble approach for early Anomaly Detection

Abstract

Proactive anomaly detection refers to anticipating anomalies or abnormal patterns within a dataset in a timely manner. Discovering anomalies such as failures or degradations before their occurrence can lead to great benefits such as the ability to avoid the anomaly happening by applying some corrective measures in advance (e.g., allocating more resources for a nearly saturated system in a data centre). In this paper we address the proactive anomaly detection problem through machine learning and in particular ensemble learning. We propose an early Anomaly Detection Ensemble approach, ADE, which combines results of state-of-the-art anomaly detection techniques in order to provide more accurate results than each single technique. Moreover, we utilise a a weighted anomaly window as ground truth for training the model, which prioritises early detection in order to discover anomalies in a timely manner. Various strategies are explored for generating ground truth windows. Results show that ADE shows improvements of at least 10% in earliest detection score compared to each individual technique across all datasets considered. The technique proposed detected anomalies in advance up to ∼16h before they actually occurred.