Applying Fourier Inspired Windows for Concept Drift Detection in Data Stream

Abstract

Detecting concept drift by mining data stream is essentially based on two parameters - (a) the window length for observing the data, and (b) the threshold for the difference in certain properties of the data observed to confirm the drift. Efforts are mostly focused on the evaluation the statistical properties or building models using the data in the window. Less focus has been given in arriving at the size of the window length. This paper presents a Fourier analysis based mechanism to determine the window length and also presents a simple but novel algorithm for drift detection. The work is termed as Fourier Inspired Windows for Concept Drift detection (FIWCD). Fourier-inspired window slides over the stream till no drift are suspected and model parameters are gradually updated as and when required. Once a drift is suspected, the trend is observed and new model parameters are computed for the transitory phase. The transitory model and previous concept model are compared to confirm or reject the drift. Performance of proposed drift detection technique has also been compared with two popular drift detection techniques namely, Change Point Detection and Hypothesis Testing. Experiment with three commonly used public data sets reflects that FIWCD exhibits better resemblance to the ground truth.