Following Fast-Dynamic Targets With Only Slow and Delayed Visual Feedback: A Kalman Filter and Model-Based Prediction Approach

Although visual feedback has enabled a wide range of robotic capabilities such as autonomous navigation and robotic surgery, low sampling rate and time delays of visual outputs continue to hinder real-time applications. When partial knowledge of the target dynamics is available, however, we show the potential of significant performance gain in vision-based target following. Specifically, we propose a new framework with Kalman filters and multirate model-based prediction (1) to reconstruct fast-sampled 3D target position and velocity data, and (2) to compensate the time delay for general robotic motion profiles. Along the path, we study the impact of modeling choices and the delay duration, build simulation tools, and experimentally verify different algorithms with a robot manipulator equipped with an eye-in-hand camera. The results show that the robot can track a moving target with fast dynamics even if the visual measurements are slow and incapable of providing timely information.