Scalable Inference via Averaged Robbins-Monro Bootstrap

Abstract

Bootstrap procedures represent a straightforward approach to assessing the uncertainty around estimates of interest in statistical models. However, with the rising prevalence of massive datasets in statistical problems, the computational cost of bootstrap methods can quickly become prohibitive in many settings. To this end, this paper proposes the Averaged Robbins-Monro Bootstrap (ARM-B), a scalable tool for estimating parameter variability via multiple chains of Robbins-Monro updates. The method is illustrated in large-scale Poisson regression and logistic regression settings and compared with the alternative scalable method given by the bag of little bootstraps (BLB). Some simulation experiments and an illustrative analysis on a large-scale dataset show that ARM-B has comparable accuracy with ordinary bootstrap, but, at the same time, it is significantly less computationally demanding and quite competitive with BLB.