Efficient hierarchical MCMC for policy search

Abstract

Many inference and optimization tasks in machine learning can be solved by sampling approaches such as Markov Chain Monte Carlo (MCMC) and simulated annealing. These methods can be slow if a single target density query requires many runs of a simulation (or a complete sweep of a training data set). We introduce a hierarchy of MCMC samplers that allow most steps to be taken in the solution space using only a small sample of simulation runs (or training examples). This is shown to accelerate learning in a policy search optimization task.