Bag-of-Visual-Phrases via Local Contexts

Abstract

This paper extends the bag-of-visual-words representations to a bag-of-visual-phrases model. The introduced bag-of-visual-phrases representation is constructed upon a proposed method for probabilistic description of co-occurring visual words, which is adapted for each reference word. This bag-of-visual-phrases representation implicitly encodes spatial relationships among visual words, thus being a richer representation while remaining as compact as the bag-of-visual-words model. We demonstrate the effectiveness of our method with a series of statistical analysis and retrieval experiments, and show that it largely outperforms previous methods for construction of bag representations. Furthermore, our method allows to query traditional bag-of-words vs the proposed bag-of-phrases. We conducted retrieval experiments on a dataset of complex shapes, whose instances correspond to hieroglyphs of the pre-Columbian Maya culture from the ancient Americas.