Enhancing Crowd Wisdom Using Explainable Diversity Inferred from Social Media

Abstract

A crowd sampled from a set of individuals can provide a more accurate prediction in aggregate than most individuals.This effect, referred to as wisdom of crowd, exists when crowd members bring diverse perspectives to decision making. Such diversity leads to uncorrelated prediction errors that cancel out in aggregate. As crowd members' judgments are often the result of solution strategies, diversity in solution strategies can enhance crowd wisdom. One of the most challenging tasks in sampling such a crowd is to determine the individual's solution strategy for a prediction problem. As participating individuals often share their perspectives through social media, we can use such data to identify an individual's solution strategy. In this paper, we propose a crowd selection approach using social media posts (tweets) indicating diverse solution strategies. We use tweet classification to identify participants' prediction strategies and categorize participants based on the binomial test to identify sets of participants that apply a similar strategy. We then form a diverse crowd by sampling participants from different sets. Using the domain of Fantasy Sports, we show that such a diverse crowd can outperform crowd selected at random and 90% of individual participants, and participant categorization schemes using word2vec. Further, we use a knowledge graph to investigate the factors forming such a diverse crowd and how these factors can lead to a better decision. Relative to bottom-up (data-driven) processes the approach presented here provides an explanation of diverse crowd behavior.