Reinforcement learning, particle filters and the EM algorithm

Abstract

We consider a parameter estimation problem for a Hidden Markov Model in the framework of particle filters. Using constructs from reinforcement learning for variance reduction in particle filters, a simulation based scheme is developed for estimating the partially observed log-likelihood function. A Kiefer-Wolfowitz like stochastic approximation scheme maximizes this function over the unknown parameter. The two procedures are performed on two different time scales, emulating the alternating `expectation' and `maximization' operations of the EM algorithm. Numerical experiments are presented in support of the proposed scheme.