Fusing optical flow and stereo disparity for object tracking

Abstract

This paper proposes a novel approach to object detection and tracking using video sensors. Two different methods are employed to retrieve depth information from images: stereopsis and depth from motion. The obtained data streams are fused yielding increased reliability and accuracy. A set of image points is tracked over time using an extended Kalman filter. The proposed algorithm clusters points of similar dynamics by analysis of the filter residuals. Experimental results are provided for synthetic as well as for natural image sequences.