Consensus-based ranking of wikipedia topics

Abstract

To improve the effectiveness of users' information seeking experience in interactive web search we hypothesize how people might be influenced when making relevance judgment decisions by introducing the Consensus Theory & Relevance Judgment Model (CT&M). This is combined with a practical path to assess the extent of difference between suggestions of current search engines versus user expectations. A user-centered, evidence-based, phenomenology approach is used to improve on Google PageRank (GPR) in two ways. The first by biasing GPR's equal navigation probability assumption using (f)actual usage stats as implicit user consensus which leads to the StatsRank (SR) algorithm. Secondly, we aggregate users' explicit ranking to derive Consensus Rank (CR) which is shown to predict individual user ranking significantly better than GPR and meta-search of modern search engines Google and Yahoo/Bing real-time. CT&M contextualizes CR, SR, and a live open online web experiment, called The Ranking Game, which is based on the August-2016 English Wikipedia corpus (12.7 million pages) and Page View Statistics for May to July 2016. Limiting this work to Wikipedia makes GPR topic-based since any Wikipedia page is focused on one topic. TREC's pooling is used to merge top 20 results from major search engines and present an alphabetized list for users' explicit ranking via drag and drop. The same platform captures implicit data for future research and can be used for controlled experiments. Our contributions are: CT&M, SR, CR, and the open online user feedback web experiment research platform.