Intensity-difference based 3D video stabilization for planetary robots

Abstract

In this paper, an algorithm to remove the camera jitter from image sequences captured by planetary robots is investigated. First, the frame to frame surface 3D motion with respect to the camera coordinate system is estimated and integrated over time. The estimation is performed by maximizing a likelihood function of the frame to frame intensity differences measured at key observation points. Then, the jitter is determined as the perspective projection of the difference between the integrated surface 3D translation and a smoothed version of it. Finally, the stabilized video is synthesized by moving the entire content of each image with a displacement vector having the same magnitude but opposite direction to the estimated jitter for that image. The experimental results with synthetic data revealed real time operation with low latency and a reduction of the jitter in a factor of 20. Experimental results with real image sequences captured by a rover platform in indoor and outdoor conditions show very reliable and encouraging stabilization results.