Making Mistakes

Abstract

From law and politics to commerce and art, algorithms are powerful sociotechnical forces. But what does it mean when algorithms “fail”? What do we learn about sociotechnical dynamics when algorithms are seen to have erred or made a mistake? Seeing algorithms as culture, I argue that algorithmic errors are constructs of intertwined computational, psychological, organizational, infrastructural, discursive, and normative forces. Through three stories of error, I show algorithmic failures as illustrations not only of algorithmic power but also of normative forces that define success, rationalize iteration, and distribute harm. Instead of seeing algorithmic errors as unavoidable parts of technological innovation or self-evident transgressions, I instead see them as evidence of how people think systems should work, and the power to declare failures, trigger fixes, and envision futures by discovering and repairing mistakes. This power to “make mistakes” is a crucial and largely understudied form of sociotechnical control.