Challenges and Opportunities for Trust Management in Crowdsourcing

Abstract

Crowd sourcing (CS) systems offer a new way for businesses and individuals to leverage on the power of mass collaboration to accomplish complex tasks in a divide-and-conquer manner. In existing CS systems, no facility has been provided for analyzing the trustworthiness of workers and providing decision support for allocating tasks to workers, which leads to high dependency of the quality of work on the behavior of workers in CS systems as shown in this paper. To address this problem, trust management mechanisms are urgently needed. Traditional trust management techniques are focused on identifying the most trustworthy service providers (SPs) as accurately as possible. Little thoughts were given to the question of how to utilize these SPs due to two common assumptions: 1) an SP can serve an unlimited number of requests in one time unit, and 2) a service consumer (SC) only needs to select one SP for interaction to complete a task. However, in CS systems, these two assumptions are no longer valid. Thus, existing models cannot be directly used for trust management in CS systems. This paper takes the first step towards a systematic investigation of trust management in CS systems by extending existing trust management models for CS trust management and conducting extensive experiments to study and analyze the performance of various trust management models in crowd sourcing. In this paper, the following key contributions are made. We 1) propose extensions to existing trust management approaches to enable them to operate in CS systems, 2) design a simulation test-bed based on the system characteristics of Amazon's Mechanical Turk (AMT) to make evaluation close to practical CS systems, 3) discuss the effect of incorporating trust management into CS system on the overall social welfare, and 4) identify the challenges and opportunities for future trust management research in CS systems.