Sequential quantile regression for stream data by least squares

Abstract

Massive stream data are common in modern economics applications, such as e-commerce and finance. They cannot be permanently stored due to storage limitation, and real-time analysis needs to be updated frequently as new data become available. In this paper, we develop a sequential algorithm, SQR, to support efficient quantile regression (QR) analysis for stream data. Due to the non-smoothness of the check loss, popular gradient-based methods do not directly apply. Our proposed algorithm, partly motivated by the Bayesian QR, converts the non-smooth optimization into a least squares problem and is hence significantly faster than existing algorithms that all require solving a linear programming problem in local processing. We further extend the SQR algorithm to composite quantile regression (CQR), and prove that the SQR estimator is unbiased, asymptotically normal and enjoys a linear convergence rate under mild conditions. We also demonstrate the estimation and inferential performance of SQR through simulation experiments and a real data example on a US used car price data set.