Diversity Analysis of Web Search Results

Abstract

Are web search results usually dominated by major websites and therefore lacking diversity? In this paper, we aim to answer this question by quantitatively modelling the diversity of search results for popular queries using two diversity measures well-studied in ecology, namely Simpson's diversity index and Shannon's diversity index. Our theoretical analysis shows how the diversity of search results is determined by the Zipfian distribution of websites. Our empirical analysis reveals that comparing Google and Bing, the former is more diverse in the top-50 search results, while the latter is more diverse in the top-10 search results.