Web-based analyses of E-journal impact: Approaches, problems, and issues

Abstract

This study1 assesses the ways in which citation searching of scholarly print journals is and is not analogous to backlink searching of scholarly e‐journal articles on the WWW, and identifies problems and issues related to conducting and interpreting such searches. Backlink searches are defined here as searches for Web pages that link to a given URL. Backlink searches were conducted on a sample of 39 scholarly electronic journals. Search results were processed to determine the number of backlinking pages, total backlinks, and external backlinks made to the e‐journals and to their articles. The results were compared to findings from a citation study performed on the same e‐journals in 1996. A content analysis of a sample of the files backlinked to e‐journal articles was also undertaken. The authors identify a number of reliability issues associated with the use of “raw” search engine data to evaluate the impact of electronic journals and articles. No correlation was found between backlink measures and ISI citation measures of e‐journal impact, suggesting that the two measures may be assessing something quite different. Major differences were found between the types of entities that cite, and those that backlink, e‐journal articles, with scholarly works comprising a very small percentage of backlinking files. These findings call into question the legitimacy of using backlink searches to evaluate the scholarly impact of e‐journals and e‐journal articles (and by extension, e‐journal authors).