Self-adaptation in a middleware for processing data streams

Abstract

Many stream-based applications share a common set of characteristics, which makes grid-based and adaptive processing desirable or even a necessity. We view the problem of flexible and adaptive processing of distributed data streams as a grid computing problem. We have been developing a middleware for processing of distributed data streams. Our system is referred to as GATES (grid-based adaptive execution on streams). It is designed to use the existing grid standards and tools to the extent possible. Because of the real-time requirements, there is a need for adapting the processing in such a distributed environment, and achieving the best accuracy of the results within the real-time constraint. For this purpose, our system flexibly achieves the best accuracy that is possible while maintaining the real-time constraint on the analysis. We have developed and implemented a self-adaptation algorithm. This paper focuses on describing and evaluating this algorithm.