Threads and Needles: A Value-Sensitive Design Approach to Online Toxicity.

Abstract

This paper engages with the problem of toxic speech online and suggests remedies inspired by the value-sensitive design literature (VSD), suggesting that the designers of online platforms should explore methods of adding friction to online conversations. Second, this paper examines a historical case of designing a communications platform to offer methods to users to inculcate norms of acceptable behavior by introducing friction into synchronous conversations. This is the case of America Online (AOL) Instant Messenger, also known as AIM, which included a feature whereby users could "warn" other users, attaching a cost to, and thus disincentivizing, certain kinds of speech. The nuances of the design of this feature make it especially well-suited as a subject of study in value-sensitive design as it seems to be the product of significant reflection and foresight by its designers. In the course of examining this case, this paper proposes two novel and generalizable processes of integrating values into the design of technology, inspired by the approach of value-sensitive design: a "method of decomposition," reconstructing a user journey in order to identify possible moments of intervention; and an iterative "Innovation-Abuse-Innovation" branching diagram, which systematizes the process of anticipating abuse cases and designing responses to them. These methods build upon recent work in the literature on operationalizing ethical values in the design process. I close by illustrating the flexibility and generalizability of these methods and speculating on how they might be applied to contemporary platforms.