Video stabilization algorithm for field robots in uneven terrain

Abstract

Field robots equipped with visual sensors have been used to automate several services. In many scenarios, these robots are tele-operated by a remote operator who controls the robot motion based on a live video feed from the robot’s cameras. In other cases, like surveillance and monitoring applications, the video recorded by the robot is later analyzed or inspected manually. A shaky video is produced on an uneven terrain. It could also be caused due to loose and vibrating mechanical frame on which the camera has been mounted. Jitters or shakes in these videos are undesired for tele-operation, and to maintain desired quality of service. In this paper, we present an algorithm to stabilize the undesired jitters in a shaky video using only the camera information for different areas of vineyard based on terrain profile. The algorithm works by tracking robust feature points in the successive frames of the camera, smoothing the trajectory, and generating desired transformations to output a stabilized video. We have tested the algorithm in actual field robots in uneven terrains used for agriculture, and found the algorithm to produce good results.