Handling query answering in crowdsourcing systems: A belief function-based approach

Abstract

Crowdsourcing is defined as an emerging computation paradigm, where the power of crowds is utilized to facilitate large scale tasks that are costly or time consuming with traditional methods. One of the most important technical challenges of crowdsourcing is quality control of workers' responses. Human factors play a key role in achieving high quality answers in crowdsourcing-based solving tasks. The most major factor is pertained to the uncertainty of workers about the responses that they provide to resolve the task at hand. On the other hand, workers may have diverse levels of expertise and skill. It is then important to take into account both the degrees of uncertainty and expertise to return the most correct reliable answer. In this paper, we propose a belief functions-based approach to achieve this goal. We conduct also some comprehensive experiments to validate the effectiveness of our proposal.