Supervised Learning from Data Streams: An Overview and Update

The literature on machine learning in the context of data streams is vast and growing. This indicates not only an ongoing interest, but also an ongoing need for a synthesis of new developments in this area. Here we reformulate the definitions of supervised data-stream learning, alongside consideration of contemporary concept drift and temporal dependence. Equipped with this, carry out a fresh discussion of what constitutes a supervised data-stream learning task; including continual and reinforcement learning; highlighting major assumptions and constraints. We carry out a fresh reconsideration of approaches and methods, with regard to their suitability to modern settings. But more than a categorization of state-of-the-art streaming methods, we provide a re-introduction to what is supervised stream learning, and our emphasis here is a survey of settings, and algorithmic settings. Our main goal is to pull theory and practice of supervised learning over data streams closer together. We conclude that practical stream learning does not mandate an online-learning regime. In the modern context, learning regimes should be selected and developed according to the factual data arrival mode, resource constraints, and maximum robustness and trustworthiness. We finish with a set of recommendations to this effect.