Unified video retrieval system supporting similarity retrieval

Abstract

We present the unified video retrieval system (UVRS) which provides the content-based query integrating feature-based queries and annotation-based queries of indefinite formed and high-volume video data. It also supports approximate query results by using query reformulation in case the result of the query does not exist. The UVRS divides video into video documents, sequences, scenes and objects, and involves the three layered object-oriented metadata model (TOMM) to model metadata. TOMM is composed of a raw-data layer for a physical video stream, a metadata layer to support annotation-based retrieval, feature-based retrieval, and similarity retrieval and a semantic layer to reform the query. Based on this model, we present a video query language which makes possible annotation-based queries, feature-based queries based on color, spatial, temporal and spatio-temporal correlation and similar queries, and consider a video query processor (VQP). For similarity queries on a given scene or object, we present a formula expressing the degree of similarity based on color, spatial, and temporal order. If there is no query result, then it will be carry out a query reformulation process which finds possible attributes to relax the query and automatically reforms the query by using knowledge from the semantic layer. We carry out performance evaluation of similarity using recall and precision.