Towards a collection-based results diversification

Abstract

We present a method that introduces diversity into document retrieval using clusters of top-m terms obtained from the top-k retrieved documents through pseudo-relevance feedback. Terms from each cluster are used to automatically expand the original query. We evaluate the effectiveness of our method using a non-traditional effectiveness evaluation method, which directly measures the level of diversification by computing the cosine similarity between top-k retrieved documents based on (i) the original query and (ii) the expanded queries. Our results indicate that we can increase diversity without compromising retrieval quality.