Performance Evaluation of Inverted Index Traversal Techniques

Abstract

Large search engines process thousands of queries per second over billions of documents, making query processing a major performance bottleneck. In this paper, we study an important class of index traversal techniques called dynamic pruning, which can efficiently reduce the query computational resources. We give brief overviews of index structure and skipping structure that used to quicken the term-centric and document centric index traversal strategies. Due to the fact that document centric is more efficient than term-centric for large indexes, we review several state-of-the-art dynamic pruning approaches based on document-centric scoring. We then present explicit experimental comparisons of the above index traversal techniques on TREC GOV2 datasets with detailed analytical conclusions. Finally, we provide the prospective future study of dynamic pruning with large scale of indexes.