Structuring the processing frameworks for data stream evaluation and application

Abstract

The following work addresses the problem of frameworks for data stream processing that can be used to evaluate the solutions in an environment that resembles real-world applications. The definition of structured frameworks stems from the need to reliably assess data stream classification methods, considering the constraints of delayed label access, the costs of their acquisition and the costs of model adaptation. The current experimental evaluation often boundlessly exploits the assumption of the immediate label access to monitor the recognition quality and adapt the methods to the changing concepts. The problem is leveraged by reviewing currently described methods and techniques for data stream processing and verifying their outcomes in simulated environment. This work defines a taxonomy of data stream processing frameworks and presents four processing schemes that link the tasks of drift detection and classification while considering a natural phenomenon of label delay. The presented research shows that classification quality is significantly affected not only by the disruptive phenomenon of concept drifts and label delay, but also by the undertaken processing scheme that describes the flow of labels in the recognition system. Considering a specific processing framework depending on real-world constraints proves to be a critical aspect of reliable and realistic experimental evaluation.