A proof-of-concept study of the challenges of assessing the impact of crime scene units’ accreditation: Lessons from a difference-in-differences analysis on police finalisation rates in Australia

Abstract

Since the 1980s, quality management mechanisms gradually emerged in forensic science as a response to the challenges faced by the discipline across the globe. However, in recent years, there has been debates about whether current quality assurance and accreditation frameworks are fit-for-purpose, in particular regarding crime scene examination. Therefore, this proof-of-concept study aims to provide some preliminary empirical insights into this question and explore some of the challenges associated with the assessment of the impact of accreditation of crime scene units under both ISO/IEC 17025 and the AS 5388 quality standards. To do so, open-sourced data drawn from the Report on Government Services (RoGS) were analysed using an exploratory difference-in-difference framework and multiple linear regression models to compare pre- and post-accreditation finalisation rates for various offences, in unaccredited and accredited Australian law enforcement agencies. The findings suggest that, based on the data available and within the methodological limitations of this study, accreditation under the ISO/IEC 17025 quality standard had mostly no significant improvement or decline in finalisation rates. In contrast, accreditation under the AS 5388 quality standard was consistently associated with significantly higher finalisation rates among accredited agencies compared to their non-accredited counterparts. While these results echo ongoing debates and uncertainties regarding the positive and/or negative impacts of accreditation and quality assurance in forensic science (if any), above all, they underscore the challenges of defining clear objectives, developing appropriate metrics, and designing robust methodological approaches for a more comprehensive empirical assessment of their effects. Although finalisation rates alone provide a limited indicator, this study represents a valuable first step toward a systematic evaluation of the impact of quality assurance and accreditation in forensic science.