Ensembles of Long Short-Term Memory Experts for Streaming Data with Sudden Concept Drift

Abstract

One of the challenges encountered when processing streaming data is a change of the data distribution, which is called concept drift. It has been shown that ensemble methods are effective in reacting to such a change. However, so far it has not been investigated how the architecture and configuration of the ensemble, as well as the properties of the scenario, influence the prediction accuracy if the ensemble members (experts) are Long Short-Term Memory networks with an internal state. This paper evaluates six ensemble architectures in several configurations with regards to their suitability for processing streaming data with sudden, recurring concept drift. The evaluation with a public dataset shows the impact of the architecture and configuration on the ensembles’ accuracies, as well as the influence of the concepts’ stability periods and the Long Short-Term Memory experts’ internal states under several conditions.