Path-based constraints for accurate scene reconstruction from aerial video

Abstract

This paper discusses the constraints imposed by the path of a moving camera in multi-view sequential scene reconstruction scenarios such as in aerial video, which allow for an efficient detection and correction of inaccuracies in the feature tracking and structure computation processes. The main insight is that for short, planar segments of a continuous camera trajectory, parallax movement corresponding to a viewed scene point should ideally form a scaled and translated version of this trajectory when projected onto a parallel plane. Two inter-camera and intra-camera constraints arise, which create a prediction of where all feature tracks should be located given the consensus information of all accurate tracks and cameras, which allows for the detection and correction of inaccurate feature tracks, as well as a very simple update of scene structure. This procedure differs from classical approaches such as factorization and RANSAC. In both aerial video and turntable sequences, the use of such constraints was proven to correct outlier tracks, detect and correct tracking drift, allow for a simple updating of scene structure, and improve bundle adjustment convergence.