Developing a Criteria Framework for Peer Review: A Critical Interpretive Synthesis

Abstract

The review criteria that reviewers and editors use are crucial in the journal peer review process. However, the review criteria for manuscripts are scattered across various literature, and their different manifestations make things more complicated. In response, we conducted a critical interpretive synthesis to provide a systematic criteria framework with clear definitions for reviewing manuscripts. We extracted review criteria from 157 heterogeneous sources, including 33 research articles, 20 literature reviews, 20 editorials and 84 reviewer guidelines from journals or publishers. The analysis of the evidence followed a ‘bottom-up’ approach. Five categories emerged (i.e., value to journal, effective use of literature, rigorousness, clarity and compliance) involving 12 components, 33 items and 79 entries. Drawing on the results, we developed a four-level criteria framework (i.e., categories-components-items-entries) for manuscript peer review. Additionally, we compared the content of review criteria across diverse fields. The findings provide a theoretical framework for standardised and systemised review criteria.