Massively Parallel Anomaly Detection in Online Network Measurement

Abstract

Detecting anomalies during the operation of a network is an important aspect of network management and security. Recent development of high-performance embedded processing systems allow traffic monitoring and anomaly detection in real-time. In this paper, we show how such processing capabilities can be used to run several different anomaly detection algorithms in parallel on thousands of different traffic subclasses. The main challenge in this context is to manage and aggregate the vast amount of data generated by these processes. We propose (1) a novel aggregation process that uses continuous anomaly information (rather than binary outputs) from existing algorithms and (2) an anomaly tree representation to illustrate the state of all traffic subclasses. Aggregated anomaly detection results show a lower false positive and false negative rate than any single anomaly detection algorithm.