Measuring the crowd: a preliminary taxonomy of crowdsourcing metrics

Abstract

Crowdsourcing initiatives benefit from tapping into diversity. A vast plethora of disparate individuals, organizations, frameworks and skillsets can all play a role in sourcing solutions to a challenge. Nevertheless, while crowdsourcing has become a pervasive phenomenon, there is a paucity of research that addresses how the crowdsourcing process is measured. Whereas research has advanced various taxonomies of crowdsourcing none to date have specifically addressed the issue of measuring either specific stages of the crowdsourcing process or the process as a whole. As a first step towards achieving this goal, this research-in-progress paper examines crowdsourcing at the operational level with a view towards (i) identifying the parts of the process (ii) identifying what can be measured and (iii) categorising operational metrics to facilitate deployment in practice. The taxonomy advanced is overarching in nature and can be deployed across disciplines. Furthermore, the preliminary taxonomy presented will offer practitioners a comprehensive list of metrics that will enable them to facilitate comparison across various crowdsourcing initiatives.