Evaluation of worker quality in crowdsourcing system on Hadoop platform

Abstract

Crowdsourcing is a new emerging distributed computing and problem solving production model on the backdrop of internet. The data size of crowdsources and tasks grows rapidly due to the rapid development of the crowdsourcing system. To evaluate the worker quality, based on the big data technology has become a more complex challenge. In this paper, we propose a general worker quality evaluation algorithm which can be applied to any critical tasks without wasting resources. Realising the evaluation algorithm in the Hadoop platform using MapReduce parallel programming is also involved. Efficiency and accuracy of the algorithm is effectively verified in the wide variety of many big data scenarios.