A sufficient condition for feature agnostic video stabilization for surveillance robots: A state space approach

Abstract

A camera is a ubiquitous sensor in surveillance robots. Often the camera is subjected to unintended motion from either external factors or inherent dynamics. The intended camera motion induces constant flow vectors for short segments of a video sequence. Video stabilization is the removal of the effect of unintended camera motion from a video sequence. The surveillance robots are often made to operate indoors, where features are scarce for motion estimation rendering traditional video stabilization methods unreliable. Unlike feature/motion extraction methods, investigation in this work is to explore the possibility of stabilizing video while remaining agnostic to the scene. One such possibility is stabilizing the video by manipulating the coefficients of a basis of the signal subspace, the smallest subspace containing the video. We show a sufficient condition for which stabilization can be performed in the signal subspace of an unstabilized video. The sufficient condition is derived by developing a novel state-space model that describes a short-length video sequence as a discrete-time state-space model. The properties of matrices corresponding to the state model are analyzed, and the corresponding results are utilized in deriving the sufficient condition. Simulation videos and experimental videos collected from a UAV validate the proposed sufficient condition.