ACCEL-RATE: a faster mechanism for memory efficient per-flow traffic estimation

Per-flow network traffic measurement is an important component of network traffic management, network performance assessment, and detection of anomalous network events such as incipient DoS attacks. In [1], the authors developed a mechanism called RATE where the focus was on developing a memory efficient scheme for estimating per-flow traffic rates to a specified level of accuracy. The time taken by RATE to estimate the per-flow rates is a function of the specified estimation accuracy and this time is acceptable for several applications. However some applications, such as quickly detecting worm related activity or the tracking of transient traffic, demand faster estimation times. The main contribution of this paper is a new scheme called ACCEL-RATE that, for a specified level of accuracy, can achieve orders of magnitude decrease in per-flow rate estimation times. It achieves this by using a hashing scheme to split the incoming traffic into several sub-streams, estimating the per-flow traffic rates in each of the substreams and then relating it back to the original per-flow traffic rates. We show both theoretically and experimentally that the estimation time of ACCEL-RATE is at least one to two orders of magnitude lower than RATE without any significant increase in the memory size.