Adaptive Load Management over Real-Time Data Streams

Abstract

Streaming applications require long-running query services against data streams. Existing data stream management systems (DSMSs) are poor at processing long-running queries with timing constrains. To address this problem, we present a real-time DSMS which can support real-time query services in unpredictable environments. In this system, long- running queries over data streams are divided into two classes: periodic and continuous queries. A mixed query model is introduced to characterize these two kinds of real-time queries. Furthermore, an adaptive load management (ALM) strategy based on dynamic execution time prediction is proposed to distribute processor time among all query instances. The objective of the ALM strategy is to provide certain guarantee on the deadline miss ratio of periodic queries and reduce the one of continuous queries, meanwhile maximizing overall query quality. A series of experiments confirm that the ALM strategy is effective in improving query quality and managing workload fluctuations.