Mobile visual search for 3-D objects: Matching user-captured video to single reference image

Abstract

We propose a three dimensional (3D) object recognition technique: it enables the user to perform "mobile visual searches for 3D objects" and get information about the real-world object that he/she is interested in. Users have only to capture a short video of the target object to realize matching to a reference image. The image-based object recognition technique recognizes an object by comparing the user-captured image(s) (hereinafter referred to as the query") to the stored reference image(s). Unfortunately, the appearance of 3D objects changes dynamically with the viewpoint. To cover the variation in appearance anticipated, we must be able to extract from the query and reference image(s) features sufficient to permit matching. We deem the traditional approach, many reference images for each object taken from different viewpoints, to be impractical. Our approach assumes that the reference image of an object is a single photo, while the query data is a video sequence from which features are extracted lor matching against the reference image. We proposed the above framework in a prior paper, and showed that it offered high accuracy when challenged with ideal data. To extend the framework such that it can handle real-world data, we propose an advanced technique that obtains the object's features from the captured video, and then matches the features to the reference image. An experiment verifies that the proposed technique can recognize real-world objects from captured videos. Its results show that the technique offers high accuracy.