Approximate Frequency Counts Algorithm for Network Monitoring and Analysis: Improvement of "Lossy Counting"

Abstract

Monitoring network traffic is important to analyze network state, so that it is necessary to observe various traffic data and then compute the data frequency counts. If we process all the incoming packets, a lot of computer resources (mainly memory and CPU) are needed. Stream mining based algorithms can approximately compute data frequency counts over data streams with small resources. Lossy Counting algorithm is one of stream mining based algorithms for data frequency counts. Although the algorithm is simple, it is able to compute only data stream of fixed length (window size) "N" given beforehand. For that reason, it is hard to transact continuous data after N-th within the same error guarantees. In this paper, we propose an algorithm for data frequency counts. We show that our algorithm can compute data frequency after sliding the window continuously with small resources.