RTSTREAM: real-time query processing for data streams

Abstract

Many real-time applications, such as traffic control systems, surveillance systems and health monitoring systems, need to operate on continuous unbounded streams of data. These applications also have inherent real-time performance requirements that have to be met under high-volume, time-varying incoming data streams. In this paper, we present a real-time data stream query model named PQuery, which provides periodic real-time queries on data streams for the aforementioned real-time applications. To support the PQuery model, a real-time data stream management prototype system named RTSTREAM is developed to provide deadline miss ratio guarantees for periodic queries over continuous and unbounded data streams. We describe the periodic query semantics and discuss why the periodic query model is appropriate for real-time applications. To handle irregular data arrival patterns and query workloads, we propose data admission as an overload protection mechanism. We conduct performance studies with synthetic workloads as well as real workloads from network traffic monitoring applications. The experimental results show that the proposed periodic query model suits the need of the real-time applications and the data admission overload protection approach is effective in managing the workload fluctuations