Constructing Recidivism Risk

Abstract

“Evidence-based sentencing” informs criminal sentencing determinations by using statistically derived risk assessment tools to predict a defendant’s likelihood of committing future crimes. By relying on data-driven risk assessment tools, this practice applies Big Data techniques to sentencing. This Article challenges the perception that such risk assessment tools neutrally classify a defendant’s recidivism risk. Scientists who construct such tools necessarily make normative choices and embed them in the tools’ design. Such choices – including how the scientists formulate the data set, how they define “recidivism” and which factors they select to create a risk tool’s underlying algorithm – all require subjective judgment calls and can introduce inadvertent bias. Rendered invisible once tools have been created, decisions about how to select, process and analyze data amount to distinct, if unintended, sentencing policy choices when judges use such risk assessment tools through evidence-based sentencing. That data scientists make such calls present three unique concerns. First, tool creators face diverging interests when exercising their discretion. Data scientists tend to make design choices based on data robustness and tool accuracy, but such interests can conflict with or even contradict sentencing policy. Second, tool creators are ill-equipped to resolve existing racial disparities in the criminal justice system, but their design choices potentially replicate and exacerbate these disparities significantly. Finally, tool creators have little incentive to disclose the policy and data choices made, leading to misuse of and misrepresentation about the value of their seemingly objective and scientifically derived information. A partial solution lies in requiring more transparency about the recidivism risk tools' design. Additionally, those with criminal justice expertise must be included in the tool design process. This Article calls for disclosure of data processing decisions and risk assessment tool assumptions, and review by trained governmental entities to translate the design choices for consumption by judges and probation officers in states that permit evidence-based sentencing. Recognizing the intricate and problematic connection between Big Data and evidence-based sentencing, this Article concludes by considering obstacles to even this modest call for oversight in such a new and sometimes inaccessible area of criminal justice reform.