A hybrid approach for image/video content representation and identification

Abstract

In this paper, we present a framework for visual content representation (signature) along with a hierarchical filtering approach for similar content identification. For each original video to store in the database, we first extract its key frames. Next, we generate a hybrid signature of which consists of a base signature; a filtering index; and SURF descriptors. Given a suspect video, its hybrid signature is created in the similar way. The hierarchical filtering approach is then applied to match the suspect video and the relevant videos in the database based on the videos' filtering index and the base signatures. However, if the process do not find a match, the query video is suspected to fall into more complicated cases such as picture-in-picture and cropped video. To tackle such cases, SURF descriptors, are then used to match. With our hierarchical filtering approach, the computation during video matching process can be reduced. We tested our system on the dataset from TRECVid 2006, and it is demonstrated that our hierarchical filtering system along with the hybrid signature framework is very effective and helps improve the overall system performance as compared to the earlier works.