Conspicuity-based visual scene semantic similarity computing for video

Abstract

Based on saliency region representation of visual scene, a framework for quantifying the semantic similarity of two video scenes is proposed in this paper. Frame-segment key-frame strategy is used to concisely represent video content in temporal domain. Spatio-temporal conspicuity model for basic visual semantics, a neuromorphic model that simulates human visual system, is used to select dynamic and static spatial salient areas. With pattern classification technique, the basic visual semantics are recognized. Then, the similarity of two visual scenes is calculated according to information theoretic similarity principles and Tversky's set-theoretic similarity. Experiment results demonstrate the framework could compute quantitative semantic similarity of two video scenes.