From textual queries to visual queries

Abstract

In this paper we present a framework to transform textual queries into visual ones. The proposed method uses standard image retrieval techniques with textual queries and the Fast Geometric Consistency Test (FGCT) method. For every textual query a set of images is retrieved and for every image a set of descriptors is extracted. Extracted features are combined with respect to their similarity in their descriptors' space and afterwards with respect to their geometric consistency on the image plane. All pairs of images are tested for consistent geometric structures using the FGCT method. This procedure extracts the subset of images that have a persistent geometric formation in the descriptors' space. Descriptors that compose the persistent formation are extracted and used as the input in a visual query; those features constitute the visual context of the visual query. Afterwards we perform again the FGCT method, but this time using the set of extracted features of the persistent formation into the cloud of images that consists of images with out a priori textual knowledge. It is noteworthy that the proposed method is scale, rotation and translation invariant. Experimental results on the Microsoft's Clickture dataset which consist of 1 million images are presented to support these statements.