Citations as a measure of the research outputs of New Zealand's economics departments: The problem of 'long and variable lags'

The paper explores the merits of utilising citation counts to measure research output in economics in the context of a nationwide research evaluation scheme: the New Zealand Performance-Based Research Fund (PBRF). Citations were collected for all refereed papers produced by New Zealand's academic economists over the period 2000 to 2008, and used to estimate the time-lags in between publication and the flow of citations; to demonstrate the impact of alternative definitions of 'economics-relevant' journals on citation counts; and to assess the impact of citation measures on departmental and individual performance. We conclude that under certain scenarios around 60 per cent of papers received no citations over the period. Our findings suggest that the time-lags between publication and citation make it difficult to rely on citation counts to produce a meaningful measure of output in a PBRF-like research-evaluation framework, especially one based on the assessment of individual academics.