Inference for High-Dimensional Streamed Longitudinal Data

Abstract

With the advent of modern devices, such as smartphones and wearable devices, high-dimensional data are collected on many participants for a period of time or even in perpetuity. For this type of data, dependencies between and within data batches exist because data are collected from the same individual over time. Under the framework of streamed data, individual historical data are not available due to the storage and computation burden. It is urgent to develop computationally efficient methods with statistical guarantees to analyze high-dimensional streamed data and make reliable inferences in practice. In addition, the homogeneity assumption on the model parameters may not be valid in practice over time. To address the above issues, in this paper, we develop a new renewable debiased-lasso inference method for high-dimensional streamed data allowing dependences between and within data batches to exist and model parameters to gradually change. We establish the large sample properties of the proposed estimators, including consistency and asymptotic normality. The numerical results, including simulations and real data analysis, show the superior performance of the proposed method.