Image retrieval using two-dimensional inverted index and semantic attributes

Most of the current image retrieval systems for large scale database rely on the Bag-of-Words (BoW) representation and inverted index. We analyze these systems and find that the retrieval performance is largely determined by the discriminative ability of their inverted indexes. This motivates us to combine SIFT and local color features into a two-dimensional inverted index (TD-II). Each dimension of TD-II corresponds to one kind of features, so the precision of visual match is enhanced. After constructing the TD-II of local features, we introduce a semantic-aware co-indexing algorithm which utilizes 1000 semantic attributes to insert similar images to the initial set of TD-II. Embedding semantic attributes into TD-II is totally off-line and effectively enhances the retrieval performance of TD-II. Experimental results demonstrate the competitive performance of our method, comparing with recent retrieval methods on two benchmark datasets, i.e., Ukbench and Holidays.