Regulating Government AI and the Challenge of Sociotechnical Design

Artificial intelligence (AI) is transforming how governments work, from distribution of public benefits, to identifying enforcement targets, to meting out sanctions. But, given AI's twin capacity to cause and cure error, bias, and inequity, there is little consensus about how to regulate its use. This review advances debate by lifting up research at the intersection of computer science, organizational behavior, and law. First, pushing past the usual catalogs of algorithmic harms and benefits, we argue that what makes government AI most concerning is its steady advance into discretion-laden policy spaces where we have long tolerated less-than-full legal accountability. The challenge is how, but also whether, to fortify existing public law paradigms without hamstringing government or stymieing useful innovation. Second, we argue that sound regulation must connect emerging knowledge about internal agency practices in designing and implementing AI systems to longer-standing lessons about the limits of external legal constraints in inducing organizations to adopt desired practices. Meaningful accountability requires a more robust understanding of organizational behavior and law as AI permeates bureaucratic routines. Expected final online publication date for the Annual Review of Law and Social Science, Volume 19 is October 2023. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.