Assuring privacy and reliability in crowdsourcing with coding

Abstract

Crowd workers are often unreliable and anonymous. Hence there is a need to ensure reliable work delivery while preserving some level of privacy to the requester's data. For this purpose, we use a combination of random perturbation to mask the sensitive data and error-correcting codes for quality assurance. We also consider the possibility of collusion attacks by malicious crowd workers. We develop mathematical models to study the precise tradeoffs between task performance quality, level of privacy against collusion attacks, and cost of invoking a large crowd. Such a study provides design strategies and principles for crowd work. The use of classification codes may improve efficiency considerably. We also comment on the applicability of these techniques for scalable assessment in education via peer grading, e.g. for massive open online courses (MOOCs).