Commentary: Prioritising Randomized Trials; David P. Farrington's Legacy for Criminology in the 21st Century

While randomized trials have become more common over the last quarter century, it remains the case that they are still rare relative to other approaches to assess programs or policies. Mazerolle et al. (2023), for example, identifies an upward trend in the use of randomized controlled trials in policing between 2004 and 2018, but shows that only 12% of all studies of evidence-based policing during that period used this method.

The increasing number of randomized trials that have been conducted in the 21st century in criminal justice suggests the practicality of carrying out randomized trials in a variety of areas in criminology. However, it is certainly still the case that many practitioners avoid randomized trials because of the constraints they often place on practitioner decision making (e.g. Clarke and Cornish 1972). It is also the case that ethical and legal concerns often come up in developing such trials, as well as practical (financial and other) challenges around implementation of RCTs. David Farrington discussed alternative approaches when randomized trials are not an option, such as opting for within-individual analyses of crime causation (as opposed to between-individual analyses) based on high quality prospective longitudinal data (Farrington et al. 2002). These can all be strong methods for drawing conclusions about programs or interventions, but Farrington saw it as critical for criminology as a science to prioritise randomized trials.

My observations of criminology as a discipline over the last few years lead me to another cause of resistance to randomized trials, which has not been noted as a key barrier to experimental research in the past. This is the turn towards ideology over science in criminology more generally. I do not have empirical data on this trend, but personal experience and conversations with colleagues lead me to the view that it has emerged as an important barrier to objective scientific work. An example of this was brought to my attention at a recent American Society of Criminology meeting at a panel I observed. One of the panelists, after noting that police and police research have become unpopular and stigmatised in criminology, stated that ‘it is clear from research that the police do not prevent crime’, continuing that only interventions relying on partnerships with the community prevent crime. I heartily agree that when the police partner with the community, they are likely to have greater crime prevention benefits. My colleagues and I have also produced evidence of this and argued for community/police partnerships over the years (e.g., see Braga and Weisburd 2010). But there is also strong experimental evidence that the police can prevent crime when they simply increase patrol at crime hot spots without community involvement (Sherman and Weisburd 1995; Braga et al. 2019). This finding is well established, having been stated in two National Academy of Sciences panels (Frydl and Skogan 2004; Weisburd and Majmundar 2018), as well as a Campbell Systematic Review (Braga 2007). But it does not fit what many criminologists want to be true about policing and crime control. This is an example of ideology overcoming science.

The concern I raise is that sometimes ideology, and what many criminologists want to believe, seems to have overcome scientific evidence. I always tell my students that ‘our job is to tell the truth’—to openly describe what the evidence says, and in this regard, to also be clear about the limitations of the evidence. Sometimes the evidence contradicts what we think is true, or what we would like to be true. But our obligation as scientists is to present the evidence, not what we would like the evidence to be. Sometimes the evidence does not fit what is popular or acceptable in a political context. This panel was just one small example of the ideological turn in criminology, but it seems to me to be part of a more general trend that places ideology before science.

I have had a number of conversations with young colleagues over the last year in which I was struck by their feelings that reviewers were evaluating their papers with a bias towards a particular outcome or perspective. Often this involved showing the effects of race or stratification on outcomes, whereby they felt they were expected to come out with a particular set of findings. It is certainly the case that race and stratification are among the most important issues that America, and indeed other societies face, but if the perspective brought by these colleagues is right, then we will be farther away from addressing these problems because of a bias towards particular outcomes. This is a point made by Pattillo (2021) who argues that social scientists have focused too much on disadvantages of black people leading to reinforcing stereotypes, and have ignored strong evidence of specific advantages of Black communities. Science is about identifying as best as we can the ‘truth’, not about reinforcing particular political positions or beliefs.

It is important to note that our research proposal, while having implications for hot spots policing, and also for social interventions focused on crime hot spots, did not involve the police at all. Rather we proposed to collect crime data at a micro geographic unit of analysis to identify levels of crime concentrations in cities and what contributes to greater or lower intensities of crime concentration. This criticism did not prevent our project from being funded, but it was ‘an important’ criticism that we needed to respond to.

What is significant here is that the reviewer felt it important to raise these issues, and also did not feel at all constrained about detailing accusations about the police which fit a specific ideological view of policing. There is certainly much research detailing police abuse, but the comments here show little nuance about the world of policing. Moreover, the reviewer did not feel at all constrained about asking us to review whether our research in the past has been applied in an ethical and procedurally just manner. This is certainly an interesting question and one that my colleagues and I have delved into in an experimental field trial (Weisburd et al. 2022) but requiring a review of past work in a proposal about something different certainly seems to be an ideological turn. Finally, the comments about enhancement of career are certainly outside the framework of a review. However, the funder clearly felt obligated to include these comments which itself raises questions about the pressure to be responsive to particular ideological perspectives in criminology.

I do not have empirical evidence of the devaluing of experimental work in criminology, but I have again heard from younger scholars who tell me that there has been a shift away from valuing strong experimental research and meta analyses of such research, towards ethnographic and qualitative research in criminology. I want to emphasise that such research is an important part of science, and in my view is complementary to randomized trials. For example, there is certainly a strong benefit to the inclusion of qualitative data in systematic reviews (Ajzenstadt 2016). But such research is not a substitute for rigorous experimental trials of programme outcomes. Qualitative methods, for example, are key to developing understandings of the mechanisms that drive experiment findings and can provide important insights into questions that need to be addressed by criminologists. And they have much value outside of experimental paradigms. But what I am hearing is more of a push to devalue experimental research. The reaction to the ‘engineers view’ of experimental research as a path to advancing knowledge about what works (Stevenson 2023) which argues that there cannot be an ‘evidence based’ approach built on experimental science, moves us to a criminology where ideology is on par with science. This directly contradicts David Farrington's legacy which called for us to prioritise experiments in evaluation research. We need to conduct rigorous experimental studies and be open to findings that depart from conventional wisdom!

Randomized trials have particular value in evaluations of programs and treatments. Of course, not all randomized trials are carried out in ways that the benefits of randomisation are fully realized. For example, sometimes the procedures are not reliably followed, and in such cases the ability of a randomized trial to provide valid answers is challenged. In turn, sometimes randomized experiments cannot be widely generalised because of the specific samples that are examined. There are clear challenges when it comes to implementation of randomized trials in real life with high fidelity and, certainly, additional challenges when it comes to nuanced interpretations of relevant study findings (Jeffery et al. 2024). However, randomized experiments carried out with fidelity and examining populations of interest provide conclusions that are less ambiguous and less threatened by researcher bias than other commonly used methods in criminology (Weisburd, Lum, and Petrosino 2001). This was a point that David Farrington made throughout his career as he tried to encourage rigorous randomized trials.

David Farrington sought to create a stronger science in criminology, in which randomized trials would be prioritised in evaluations of programs and policies. This does not mean that he saw randomized trials as the only approach that should be valued. He was a strong supporter of longitudinal research and argued that when randomised trials are not an option, other analyses that mimic experiments—including within-individual analyses and propensity score matching—should be promoted in trying to establish the causes of crime (Farrington et al. 2020). In the Babylonian Talmud, a sixth century compilation of Jewish law and ethics, we are told: ‘That you are not obligated to [complete the repair of the world], but you are not free to not contribute to that effort’ (Ethics of the Fathers 2:16). While there remains much work to be done, David Farrington's work has played a key part in advancing science in criminology.

The author declares no conflicts of interest.