Crowd-Sourcing for Data Science and Quantifiable Challenges: Optimal Contest Design

Abstract

We study the optimal design of a crowd-sourcing contest in settings where the output (from the contestants) is quantifiable -- for example, a data science challenge. This setting is in contrast to settings where the output is only qualitative and cannot be quantified in an objective manner -- for example, when the goal of the contest is to design a logo. The rapidly growing literature on the design of crowd-sourcing contests focuses largely on ordinal contests -- these are contests where contestants' outputs are ranked by the organizer and awards are based on the relative ranks. Such contests are ideally suited for the latter setting, where output is qualitative. For our setting (quantitative output), it is possible to design contests where awards are based on the actual outputs and not on their ranking alone -- thus, our space of contest designs includes ordinal contests but is significantly larger. We derive an easy-to-implement contest design for this setting and establish its optimality.