Distribution Estimation for Stochastic Approximation in Finite Samples Using A Surrogate Stochastic Differential Equation Method

Evaluating the statistical error in the estimate coming from a stochastic approximation (SA) algorithm is useful for confidence region calculation and the determination of stopping times. Robbins-Monro (RM) type stochastic gradient descent is a widely used method in SA. Knowledge of the probability distribution of the SA process is useful for error analysis. Currently, however, only the asymptotic distribution has been studied in this setting in asymptotic theories, while distribution functions in the finite-sample regime have not been clearly depicted. We developed a method to estimate the finite sample distribution based on a surrogate process. We described the stochastic gradient descent (SGD) process as a Euler-Maruyama (EM) scheme for some RM types of stochastic differential equations (SDEs). Weak convergence theory for EM schemes validates its surrogate property with a convergence in distribution sense. For the first time, we have shown that utilizing the solution of Fokker-Planck (FP) equation for the surrogate SDE is appropriate to characterize the evolution of the distribution function in SGD process.