CrowdMerge: Achieving Optimal Crowdsourcing Quality Management by Sequent Merger

Quality management of worker responses is an increasingly important issue in crowdsourcing, which can be a fundamental reference for workload allocation. However, most of the existing solutions to worker response evaluation meet with significant challenges. Part of them are only able to obtain locally optimal results with low accuracy, while the others have to pay a high price to achieve global optimality. In order to tackle these challenges, this paper proposes a solution to worker quality management which is able to obtain optimal evaluation results close to the globally one with acceptable computational overhead. It is proved to be globally optimal results under certain conditions. The basic idea of the approach is to calculate the similarities among the workers according to which the workers can be clustered gradually. The unknown answers (or outputs) of the tasks are predicted from the results obtained by the clustered workers which will further help with the similarity calculation and the clustering among the remaining workers. A framework and detailed schemes are presented, and simulation experiments are conducted to validate the efficacy of the approach.