Metrics as Proxies: A Theoretical and Empirical Study of Reasonings Based on Bibliometric Indicators

Abstract

Increasingly, reviewers are asked to engage with citation-based indicators, yet we know little about how those numbers are actually translated into judgements. This article builds on an extensive literature review in which the author identifies “chains of reasoning” that connect metrics with factors such as quality, prestige, recognition, and career potential. The study analyses 174 negative habilitation review reports (2012–2021) from four disciplines in Poland—Art History, Sociology, Mathematics, and Materials Engineering—to map how reviewers reason with metrics in practice. We identify and code passages where quantitative indicators are transformed into qualitative claims and reconstruct the “chains of reasoning” that link numbers to evaluative conclusions. The prevalence of metric-based reasoning varies sharply by field—from about 4% of reviews in Art History to 71% in Materials Engineering—while Mathematics and Sociology occupy intermediate positions. Across disciplines, metrics are interpreted primarily as signals of recognition or visibility and influence or uptake; direct inferences from metrics to intrinsic quality are comparatively rare. Reviewers sometimes inspect citation contexts (e.g., self-citations) and time lags, but explicit reflexivity about the limitations of indicators is infrequent. Crucially, metrics never stand alone: even the most metric-intensive reviews combine numbers with content-based critique. We catalogue nine recurring reasoning types and show how a uniform policy that mandates attention to metrics yields heterogeneous, discipline-specific evaluative practices. The article advances scholarship on responsible research assessment by explaining how metrics travel from numbers to judgements and by offering practical guidance: use indicators as contextual evidence, make interpretive assumptions explicit, and calibrate benchmarks to disciplinary evaluative cultures.