The limitations and future of violence risk assessment

Abstract

Laws to protect the public from mentally ill people who have committed a violent offence date from the attempted assassination of King George III by a disturbed ex-soldier in 1800. In the last 50 years, the assumption that mental illness is both a cause and a predictor of violence has led to changes in mental health laws that limit involuntary treatment to those considered to be dangerous and to research into how to assess the risk of violence. The most common form of violence risk assessment is still a judgment made by a clinician. However, this form of assessment lacks transparency, is vulnerable to cognitive biases and relies on the experience and expertise of the clinician. Actuarial assessments based on a score from of a list of identified risk factors have made violence risk assessment more objective, reliable and probably more accurate. More than 200 actuarial violence risk instruments have been described. Despite their advantages over unaided clinical judgment, there are both scientific and ethical problems with the use of these instruments in clinical practice. The scientific concerns are about the strength of the statistical separation of high-risk and lower-risk groups, the overreliance on measures of discrimination (such as the area under the curve or odds ratios) rather than measures of prediction (such as the positive predictive value), the applicability of instruments to different groups, and the extent to which aggregate risk data apply to individuals. The ethical concerns include the potential for risk assessment to add to the stigma and discrimination experienced by the mentally ill, unfair restrictions after false positive predictions, and denial of care to those assessed to be lower-risk. With these concerns in mind, any evaluation of the current state of violence risk assessment must answer two important questions: Does violence risk assessment produce valid information? And is this information clinically useful? The first question has been answered by a recent metaanalysis of 92 studies that independently replicated the results of nine popular violence risk instruments. The pooled estimate of the diagnostic odds of violence among high-risk patients was 3.08 (95% CI: 2.45-3.88), indicating that the rate of severe violence can be expected to be about three times higher in high-risk groups than lower-risk ones. An odds ratio of three indicates that risk assessment produces valid information with a modestly strong effect size – a degree of separation between high-risk and lower-risk groups similar to the risk of suicide associated with male gender. To answer the second question about the usefulness of the information generated by a violence risk assessment, we need to consider whether there are treatments or interventions that can be reasonably allocated to high-risk patients but denied to lower-risk patients, and whether the transfer of treatment resources from lower-risk to high-risk groups actually reduces the overall rate of violence. Intervening on the basis of a score generated by a violence risk instrument can only be reasonable if the proportion of patients correctly predicted (true positives) is sufficiently high to justify the treatment of all those at high risk (true and false positives). Hence, risk guided interventions must be both effective and benign, because the low base rates for serious violence means that there will always be many false positives for every true positive prediction. Moreover, even if there is the opportunity to prevent some episodes of severe violence, interventions guided by the results of risk assessment can only be justified if there is a compelling reason for not intervening in lower-risk patients, who inevitably commit a proportion of all violent acts. Few interventions meet this test, which might explain why, among the thousands of publications about risk assessment, there are as few as three controlled studies of risk guided interventions that have rates of violence as an outcome measure. The time has come to shift the debate away from arguments about the numerical properties of violence risk instruments towards a consideration of whether being able to identify individuals with a greater risk can actually result in a reduction in the overall rate or severity of violence. A few controlled trials of the violence reducing properties of risk guided interventions would produce more useful information than any number of studies of the predictive properties of violence risk instruments. What then is the future of violence risk assessment? Incremental improvements in predictive accuracy might follow the discovery of new risk factors or new ways of combining established risk factors using more sophisticated statistical techniques, or a reduced reliance on historical factors and a greater emphasis on the person’s current situation. In the future, violence risk assessment is likely to shift from cross-sectional prediction to ongoing clinical monitoring, using technology such as the analysis of social media and even telemetry reporting physiological markers of intoxication and abnormal mood states. We might tolerate some increased intrusion into the lives of our patients if new methods are shown to be effective in reducing violence. However, any new methods should not only be assessed by their predictive ability, but also by reliable evidence that they can actually reduce violence and that any reduction is not at an unacceptable cost to an already disadvantaged section of society.