Concept Drift Detection Based on Pre-Clustering and Statistical Testing

Abstract

Stream data processing has become an important issue in the last decade. Data streams are generated on the fly and possibly change their data distribution over time. Data stream processing requires some mechanisms or methods to adapt to the changes of data distribution, which is called the concept drift. Concept drift detection can be challenging due to the data labels are not known. In this paper, we propose a drift detection method based on the statistical test with clustering and feature extraction as preprocessing. The goal is to reduce the detection time with principal component analysis (PCA) for the feature extraction method. Experimental results on synthetic and real-world streaming data show that the clustering preprocessing improve the performance of the drift detection and feature extraction trade-off an insignificant performance of detection for speedup for the execution time.