Towards fair and pro-social employment of digital pieceworkers for sourcing machine learning training data

Abstract

while there are bad actors, the distributed nature of the platforms may cause requesters to appear to enact these malpractices out of misunder-standing more than malintent. For this purpose, we propose a cover sheet describing precise hiring and employment practices of academic 1 collaborators (or “crowd collaborators” ) engaged through crowd-working platforms. The cover sheet is to be submitted with the publication of the resulting work by researchers in academic venues. The design of the cover sheet is inspired by the Datasets for Datasheets project [13], to be required by academic venues accepting the results of crowd collaborator labor, namely machine learning training datasets. As Hießl argues, crowd collaborators do not have traditional employment contracts to rely on and that a new form of contract must be developed to address the complexity of digital piecework [16]; we present this cover sheet as a frst step in that direction. By surfacing this information at the institutional level we hope to 1) inform requesters of the best practices if they are unaware, and 2) certify respectful treatment of crowd collaborators, especially given the calls to substitute digital piecework for jobs lost in the face automation [20, 29]. Our intervention centers academic requesters for two reasons. First, we choose to highlight the role of requesters in these platforms as the power balance is inevitably shifted away from those performing the labor to those providing it, due to the oversupply of ABSTRACT This pieceworkers ("crowd collaborators") by reforming the handling of crowd-sourced labor in academic venues. With the rise in automation, crowd collaborators treatment requires special consideration, as the system often dehumanizes crowd collaborators as components of the “crowd” [41]. Building of eforts to (proxy-)unionize crowd workers and facilitate employment protections on digital piecework platforms, we focus on employers: academic requesters sourcing machine learning (ML) training data. We propose a cover sheet to accompany submission of work that engages crowd collaborators for sourcing (or labeling) ML training data. The guidelines are based on existing calls from worker organizations (e.g., Dynamo [28]); professional data workers in an alternative digital piecework organization; and lived experience as requesters and workers on digital piecework platforms. We seek feedback on the cover sheet from the ACM community. work contributes to just and pro-social treatment of digital