A query model for retrieving relevant intervals within a video stream

Abstract

The nature of video is such that even an hour long video data may contain a large number of meaningful intervals. Manual identification of all such intervals is practically infeasible. There has been some success in automatically parsing and indexing video data through the integration of technologies such as image processing, speech/character recognition, and natural language understanding. However, even by applying such techniques, complete identification of all the intervals required for answering all possible queries cannot be achieved. As a result, using the current state-of-art techniques, whether automatic or manual, it is only fragmentary video intervals that can be successfully indexed. Our goal is to retrieve meaningful intervals within such fragmentarily indexed video streams. We propose a new set of algebraic operations which enable us to compose all the intervals that are conceivably relevant to a query. Since these operations may compose even irrelevant intervals, we provide a mechanism to exclude as many of them as possible from the answer set.