An evaluation method of academic output that considers productivity differences

Abstract

There are productivity differences among academic fields. Researchers who work in academic fields that have low productivity are pressured to publish more, and this policy may cause researchers to publish more in journals that have lenient standards and publish articles that are not necessarily valuable for their academic field. The problem is not solved by normalizing journals’ impact factors by the subjects because the normalized impact factors do not reflect the difficulty of publication in that subject. In this paper, we propose an evaluation method –Reference Group Similarity Index-that addresses the productivity differences issue. The method uses the publications of a reference group of departments that are believed to have the right publication incentives. Then, other departments are evaluated to the degree that their publications are similar to that of the reference group. We apply the method to the top 50 economics departments according to USNews rankings and show that the department rankings that we get from the Reference Group Similarity Index are largely consistent with the USNews Rankings.