To let content be or not be: Understanding the decision-making process of content moderators on social media platforms

Abstract

Content moderation plays a pivotal role in structuring online speech, but the human labour and the everyday decision-making process in content moderation remain underexamined. Informed by in-depth interviews with 16 content moderators in India, in this research, we analyse the decision-making process of commercial content moderators through the concept of sensemaking. We argue that moderation decisions are made in the context of the industry’s plural policies and efficiency requirements. An interplay of four cognitive processes of pattern identification, subjective perceptions, shared knowledge, and process optimization influences the final judgement. Once sense is enacted in the decision-making process, the sensibilities are retained by the adept moderator for future moderation decisions. Visibilizing the labour process behind commercial content moderation, we argue that everyday moderation decisions unfold in a socio-technical and economic assemblage wherein decisions are decontextualised and plausibility driven rather than consistency driven.