Recommend-Me: recommending query regions for image search

Abstract

In typical image retrieval systems, to search for an object, users must specify a region bounding the object in an input image. There are situations that the queried region does not have any match with regions in images of the retrieved database. Finding a region in the input image to form a good query, which certainly returns relevant results, is a tedious task because users need to try all possible query regions without prior knowledge about what objects are really existed in the database. This paper presents a novel recommendation system, named Recommend-Me, which automatically recommends good query regions to users. To realize good query regions, their matches in the database must be found. A greedy solution based on evaluating all possible region pairs, given a pair is formed by one candidate region in the input image and one region in an image of the database, is infeasible. To avoid that, we propose a two-stage approach to significantly reduce the search space and the number of similarity evaluations. Specifically, we first use inverted index technique to quickly filter out a large number of images having insufficient similarities with the input image. We then propose and apply a novel branch-and-bound based algorithm to efficiently identify region pairs with highest scores. We demonstrate the scalability and performance of our system on two public datasets of over 100K and 1 million images.