A High-Performance Sketch With Dynamic Memory Allocation for Priority-Oriented Data Stream Processing

Abstract

Sketch is widely used in many traffic estimation tasks due to its good balance among accuracy, speed, and memory usage. In scenarios with priority flows, priority-aware sketch, as an emerging method, provides differentiated detection accuracy for flows of different priorities, optimizing resource allocation and improving the detection accuracy of high-priority flows. However, existing priority-aware sketches methods struggle to effectively handle the dynamic changes in flow priority distribution in real-world detection environments, leading to wasted or insufficient storage space. To address this issue, this paper proposes a new priority-aware sketch with Dynamic Memory Allocation called DMA-Sketch. It dynamically adjusts the detection framework based on flow priority distribution information and adaptively allocates appropriate memory space to each storage region. The experimental results show that DMA-Sketch improves the overall priority accuracy, high-priority accuracy and throughput by up to 1.33$\times$, 16.39$\times$ and 1.88$\times$, respectively, under the scenarios with changing flow priority distribution over the state-of-the-art schemes.