On the video quality captured by a surveillance mobile robot

Abstract

In many robot surveillance applications, the major contributor to the quality degradation of the captured video is the unintended relative motion between the camera and the scene. This unintended motion induces an unintended effect called jitter in the captured video sequence. The evaluation of video quality captured by a mobile robot in surveillance scenarios is often application-specific and is often based on the amount of jitter obtained through feature tracking or camera path reconstruction or intensity patterns obtained across video frames. The contributions in this paper are two folds: development and benchmarking of a novel algorithm for video quality assessment, and jitter-specific recommendations for stabilization approaches. Unlike existing Video Quality Assessment (VQA) scores, the proposed Topology Score (TS) is a non-reference technique that does not involve feature tracking or camera path reconstruction, suitable for mobile robots used for surveillance. We adopt sliding window geometry using persistent homology concept for quantifying the jitter associated with the periodic/quasiperiodic oscillations induced by the moving mobile robots, which in turn gives a VQA score. The experimental results suggest that the trend of the proposed score aligns with the existing rhythm scores that correlate highly with human subjective evaluation, but needs reference video for the assessment. Additionally, we perform a comparative study on various video stabilization algorithms on three categories of robots based on the jitter characteristics: (1) Spherical robot videos with second-order damped oscillations causing low-frequency high-amplitude jitters, (2) Autonomous drone videos with intermittent jitters, and (3) Humanoids mimicked by the casual movements of hand-held video recorder (gait motions have a periodic structure) that contain high-frequency low-amplitude jitters from the recorder’s movement, using the proposed and existing VQA scores. We apply seven different stabilization approaches to the selected robot categories and quantify the tested algorithms’ output quality and resource requirements. Finally, we report the decision matrix based on the robot’s available resources to readily use the state-of-the-art stabilization methods in mobile robot surveillance. Our findings show that the proposed topology score is most suitable for evaluating videos captured by mobile robots in unknown environments due to non-reference assessment associated with periodic/quasiperiodic jitter, and the decision matrix to select a video stabilization algorithm based on the jitter characteristics of mobile robot for the quality improvement of captured video.