How Do Algorithmic Decision‐Making Systems Used in Public Benefits Determinations Fail? Insights From Legal Challenges

Abstract

When algorithmic decision‐making systems fail to function as intended, they become conduits for administrative error and risk producing arbitrary determinations through the very technologies meant to prevent them. Analysis of 71 federal and state court dockets contesting algorithm‐based determinations in disability, unemployment, and nutrition assistance programs shows how this risk manifests in practice. Findings show that deviations from legally prescribed outcomes occur when the translation of statutory requirements into computational logic is compromised by flawed data, problematic design choices, or inherent system limitations. These algorithmic administrative errors are neither isolated glitches nor purely technical problems; they constitute a systemic governance problem that cuts across legal, organizational, and technical domains. Addressing them requires coordinated oversight across all three areas, rather than reliance on post hoc troubleshooting.