Toward hardware support for a flexible sketch-based network traffic monitoring system

Abstract

Sketch-based data streaming algorithms are capable of summarizing high-speed network traffic in real-time without packet sampling. Therefore, they are used in many networking applications to make accurate estimates of traffic flow statistics. However, fixed hardware implementation of the sketch counters limits the flexibility of re-using the same data structure for various monitoring applications. In this paper, we propose a generic scheme for collecting flow-level data using high-speed memories on the NetFPGA-10G platform. The purpose is to enable sketch-based algorithms making statistics measurements in various network monitoring applications in a flexible manner. Experiments using real Internet traces are conducted for entropy estimation, superspreader detection, and change detection applications. The results show that the proposed scheme can support flexible selection of keys and counters for the monitoring applications achieving high measurement accuracy.