The ghost of rankings past - The lasting harmful impact of journal rankings, and what we should do instead

Abstract

Most Australian law academics will recall (many with horror) the journal ranking exercise that took place in 2008-2010. The aim then was to produce a journal ranking list with international scope and validity. Unsurprisingly, this grandiose result was not achieved and the process was abandoned in 2011 – but not before damage was done. The then Minister for Innovation, Industry, Science and Research (Senator Kim Carr) noted:"There is clear and consistent evidence that the rankings were being deployed inappropriately within some quarters of the sector, in ways that could produce harmful outcomes, and based on a poor understanding of the actual role of the rankings. One common example was the setting of targets for publication in A and A* journals by institutional research managers." The urge to rank – and to use ranking in the assessment of quality – remains strong in Australian law schools. There is a risk this discredited process will be given new impetus following the release of the latest Australian journal ranking list involving law; this time by the Australian Business Deans Council (ABDC). The ABDC describes itself as ‘the authoritative and collective voice of pro vice-chancellors, executive deans and heads of all 40 business faculties and schools in Australia’. Its list ranks 2,767 different journal titles, including selected law journals. The ranking method used involved, ‘public submissions, qualitative and quantitative data assessment, public exposure feedback and international expert consultation’. Rankings like these should be formally abandoned for at least three reasons. First, as the CALD/ERA process demonstrated, the ranking process is practically difficult. It is arbitrary and infected by subjective opinion. That leads to perverse outcomes for legal academics whose career advancement is tied to this uncertain standard. Secondly, the urge to rank is the product of a damaging misconception of the proper role of legal academics and the legal academy. The rankers appear to see law as another social science rather than a professional discipline. Thirdly, the rankings have in any event been rendered redundant by the recent decision to deregulate fees. In the brave new world, markets will provide surprisingly nuanced and rigorous measures of quality that cannot be captured in a crude ranking. We explore these reasons below.