Mining Frequent Flows Based on Adaptive Threshold with a Sliding Window over Online Packet Stream

Abstract

Traffic measurement is an important component of network applications including usage-based charging, anomaly detection and traffic engineering. With high-speed links¿the main problem with traffic measurement is its lack of scalability. Aiming at circumvent this deficiency, we develop a novel and scalable sketch to mine frequent flows over online packet stream. Dividing the sliding window into buckets, the sketch can not only be easily-implemented, but also remove obsolete data to identify recent usage trends. Besides, an unbiased estimator is introduced based on a pruning  function to preserve large flows. In particular, we illustrate a mechanism of configuring adaptive thresholds which are bound to the actual data without artificial behavior. The adaptive threshold can be regulated to target the mean number of the reserved flows in order to protect memory resources. Experiments are also conducted based on real network traces. Results demonstrate that the proposed method can achieve adaptability and controllability of resource consumption without sacrificing accuracy.