3D object tracking using shape-encoded particle propagation

Abstract

We present a comprehensive treatment of 3D object tracking by posing it as a nonlinear state estimation problem. The measurements are derived using the outputs of shape-encoded filters. The nonlinear state estimation is performed by solving the Zakai equation, and we use the branching particle propagation method for computing the solution. The unnormalized conditional density for the solution to the Zakai equation is realized by the weight of the particle. We first sample a set of particles approximating the initial distribution of the state vector conditioned on the observations, where each particle encodes the set of geometric parameters of the object. The weight of the particle represents geometric and temporal fit, which is computed bottom-up from the raw image using a shape-encoded filter. The particles branch so that the mean number of offspring is proportional to the weight. Time update is handled by employing a second-order motion model, combined with local stochastic search to minimize the prediction error. The prediction adjustment suggested by system identification theory is empirically verified to contribute to global stability. The amount of diffusion is effectively adjusted using a Kalman updating of the covariance matrix. WE have successfully applied this method to human head tracking, where we estimate head motion and compute structure using simple head and facial feature models.