On the Evaluation of Background Subtraction Algorithms without Ground-Truth

Abstract

In video-surveillance systems, the moving objectsegmentation stage (commonly based on backgroundsubtraction) has to deal with several issues like noise,shadows and multimodal backgrounds. Hence, its failureis inevitable and its automatic evaluation is a desirablerequirement for online analysis. In this paper, we proposea hierarchy of existing performance measures not-basedon ground-truth for video object segmentation. Then, fourmeasures based on color and motion are selected andexamined in detail with different segmentation algorithmsand standard test sequences for video objectsegmentation. Experimental results show that color-basedmeasures perform better than motion-based measures andbackground multimodality heavily reduces the accuracy ofall obtained evaluation results