The original sin of crowd work for human subjects research

Abstract

Purpose Academic scholars have leveraged crowd work platforms such as Amazon Mechanical Turk for human subjects research for almost two decades. However, few scholars have reflected or questioned this mode of academic research. This paper aims to examine three fundamental problems of crowd work and elaborates on their lasting effects on impacting the validity and quality of human subjects research on crowd work. Design/methodology/approach` A critical analysis is conducted on the characteristics of crowd work, and three fundamental problems of crowd work since its origin were identified, namely, the position of “Human-as-a-service,” the confusion of terminology and crowd work platforms’ abdication of responsibilities. Findings This paper explains that the three identified fundamental problems of crowd work render at least two lasting problems in crowd work-based research: first, the negligence of the teleological difference between crowd work and academic research; second, the ontological schism between scholars and institutional review boards (IRBs) in their ethical concerns and practices. Originality/value This paper critiques the foundation of crowd work-based research that has become growingly popular, extolled and taken for granted. Such a critique is deficient in literature and may seem a bit peculiar. However, we hold that it is time to take research ethics seriously in crowd work because we need to introspect and question ourselves as scholars: What is our motive or ethical stance in using crowd work for human subjects research? Is it for advancing scientific knowledge, promoting crowd workers’ welfare, or predominantly for benefiting ourselves from the fast, cheap and “good” data via crowd work?