Effective Space Saving

In computer networks, it is important to analyze the traffic and provide insights about flows sending packets through the network, e.g., to prevent overloads and DDoS attacks. In this paper, we introduce Effective Space Saving (ESS), a novel algorithm for Top-K identification, a fundamental problem in network monitoring and management. ESS can identify Top-K flows in the network and answer queries regarding flows’ frequency estimation while guaranteeing a small error and a small memory footprint. ESS tracks the frequency of only a small portion of the flows in two tables. Each entry in these tables records a mapping from a given flow id to its current frequency counter. Of these two tables, the Main table stores flows that are suspected of being the heaviest in the stream in terms of their frequency. The Window table stores other recently observed flows that are contending to enter the Main table. We use a probabilistic eviction mechanism for the Window table that is based on the collected statistics. These mechanisms improve the overall memory to accuracy tradeoff of ESS compared to other known approaches. We demonstrate the effectiveness of ESS on real and synthetic packet traces with varying degrees of skew levels. For different skews, ESS identifies the Top-K flows with smaller frequency error by a factor of between 102 to 105 compared to Space Saving [19] and by a factor of up to 10 compared to RAP [5], the two state of the art competing algorithms.