Statistical inference for smoothed quantile regression with streaming data

Abstract

In this paper, we tackle the problem of conducting valid statistical inference for quantile regression with streaming data. The main difficulties are that the quantile regression loss function is non-smooth and it is often infeasible to store the entire dataset in memory, rendering traditional methodologies ineffective. We introduce a fully online updating method for statistical inference in smoothed quantile regression with streaming data to overcome these issues. Our main contributions are twofold. First, for low-dimensional data, we present an incremental updating algorithm to obtain the smoothed quantile regression estimator with the streaming data set. The proposed estimator allows us to construct asymptotically exact statistical inference procedures. Second, within the realm of high-dimensional data, we develop an online debiased lasso procedure to accommodate the special sparse structure of streaming data. The proposed online debiased approach is updated with only the current data and summary statistics of historical data and corrects an approximation error term from online updating with streaming data. Furthermore, theoretical results such as estimation consistency and asymptotic normality are established to justify its validity in both settings. Our findings are supported by simulation studies and illustrated through applications to Seoul’s bike-sharing demand data and index fund data.