A memory-compact and fast sketch for online tracking heavy hitters in a data stream

Network traffic measurement is important for network management, including bandwidth management to mitigate network congestion, and security management to detect DDOS attacks and worm spreading. However, with the explosive volume of network data and the fast transmission speed of network packets (in giga or even tera bps), it is a challenging task to measure the size of each network flow both accurately and memory-efficiently, using the size-limited SRAM memory of line card. Therefore, many sublinear space algorithms for processing data streams have been proposed, such as CountMin (CM), Count Sketch (CS) and Virtual Active Counters (VAC), which achieve extreme memory compactness by providing probabilistic guarantees on flow size measurement accuracy. However, these existing algorithms can still be greatly improved as to the performance of both online recording and querying the per-flow size, which is needed for online tracking heavy hitters. Our paper proposes a highly compact and efficient counter architecture, called CountMin virtual active counter (CM-VAC), which provides more accurate measurement results than CM and CS under a very tight memory space. We also achieve higher query speed than VAC by modifying its query policy. We demonstrate the superior performance of our algorithm by both experimental results and theoretical analysis based on CAIDA network traces.