Task Lockouts Induce Crowdworkers to Switch to Other Activities

Abstract

Paid crowdsourcing has established itself as a useful way of getting work done. The availability of large, responsive pools of workers means that low quality work can often be treated as noise and dealt with through standard data processing techniques. This approach is not practical in all scenarios though, so efforts have been made to stop poor performance occurring by preventing satisficing behaviours that can compromise result quality. In this paper we test an intervention -- a task lockout -- designed to prevent satisficing behaviour in a simple data-entry task on Amazon Mechanical Turk. Our results show that workers are highly adaptable: when faced with the intervention they develop workaround strategies, allocating their time elsewhere during lockout periods. This suggests that more subtle techniques may be required to substantially influence worker behaviour.