Wisdom of Crowds in Operations: Forecasting Using Prediction Markets

Abstract

Prediction is an important activity in various business processes, but it becomes difficult when historical information is not available, such as forecasting demand of a new product. One approach that can be applied in such situations is to crowdsource opinions from employees and the public. Our paper studies the application of crowd forecasting in operations management. In particular, we study how efficient crowds are in estimating parameters important for operational decisions that companies make, including sales forecasts, price commodity forecasts, and predictions of popular product features. We focus on a widely adopted class of crowd-based forecasting tools, referred to as prediction markets. These are virtual markets created to aggregate crowds' opinions and operate in a way similar to stock markets. We partnered with Cultivate Labs, a leading company that provides a prediction market engine, to test the forecast accuracy of prediction markets using the firm's data from its public markets and several corporate prediction markets, including a chemical company, a retail company and an automotive company. Using information extracted from employees and public crowds, we show that prediction markets produce well-calibrated forecasting results. In addition, we run a field experiment to study the conditions under which groups work well. Specifically, we explore how group size plays a role in the accuracy of the forecast and find that large groups (e.g., 18 participants) perform substantially better than smaller groups (e.g., 8 participants), highlighting the importance of group size and quantifying the right sizes needed to produce a good forecast using such mechanisms.