Artificial Intelligence, Rationalization, and the Limits of Control in the Public Sector: The Case of Tax Policy Optimization

In this paper, we first frame the use of artificial intelligence (AI) systems in the public sector as a continuation and intensification of long-standing rationalization and bureaucratization processes. Drawing on Weber, we understand the core of these processes to be the replacement of traditions with instrumental rationality, that is, the most calculable and efficient way of achieving any given policy objective. Second, we demonstrate how much of the criticisms, both among the public and in scholarship, directed towards AI systems spring from well-known tensions at the heart of Weberian rationalization. To illustrate this point, we introduce a thought experiment whereby AI systems are used to optimize tax policy to advance a specific normative end: reducing economic inequality. Our analysis shows that building a machine-like tax system that promotes social and economic equality is possible. However, our analysis also highlights that AI-driven policy optimization (i) comes at the exclusion of other competing political values, (ii) overrides citizens’ sense of their (non-instrumental) obligations to each other, and (iii) undermines the notion of humans as self-determining beings. Third, we observe that contemporary scholarship and advocacy directed towards ensuring that AI systems are legal, ethical, and safe build on and reinforce central assumptions that underpin the process of rationalization, including the modern idea that science can sweep away oppressive systems and replace them with a rule of reason that would rescue humans from moral injustices. That is overly optimistic: science can only provide the means – it cannot dictate the ends. Nonetheless, the use of AI in the public sector can also benefit the institutions and processes of liberal democracies. Most importantly, AI-driven policy optimization demands that normative ends are made explicit and formalized, thereby subjecting them to public scrutiny, deliberation, and debate.