Model-Based 3D Object Tracking Using an Extended-Extended Kalman Filter and Graphics Rendered Measurements

This paper presents a model-based 3D object tracking system that uses an improved Extended Kalman filter (EKF) with graphics rendering as the measurement function. During tracking, features are automatically selected from the input images. For each camera, an estimated observation and multiple perturbed observations are rendered for the object. Corresponding features are extracted from the sample images, and their estimated/perturbed measurements are acquired. These sample measurements and the real measurements of the features are then sent to an extended EKF (EEKF). Finally, the EEKF uses the sample measurements to compute high order approximations of the nonlinear measurement functions, and updates the state estimate of the object in an iterative form. The system is scalable to different types of renderable models and measureable features. We present results showing that the approach can be used to track a rigid object, from multiple views, in real-time.