A nonparametric adaptive sampling strategy for online monitoring of big data streams

Abstract

With the rapid development of sensor techniques, we often face the challenges of monitoring big data streams in modern quality control, which consist of massive series of real-time, continuously and sequentially ordered observations. For example, in manufacturing industries, hundreds or thousands of variables are observed during online production for quality insurance. Also, smart grid infrastructure needs to simultaneously monitor massive access points for intrusion and threat detection. As another example, an image sensing device continuously collects high-resolution images at high frequency for video surveillance and object movement tracking. Ideally, in those applications, it is preferable to detect assignable causes as early as possible, while maintaining a prespecified in-control Average Run Length (ARL).