Peer review reports of randomized controlled trials in oncology can be short and superficial.

Abstract

Objectives: To evaluate the quality of open peer review reports published alongside articles of randomised controlled trials (RCTs) in oncology.

Methods: We searched and sampled from completed parallel RCT articles published in 2021 in 62 BioMed Central journals operating open peer review and evaluated their first-round peer review report. We assessed and described the peer review report content, clarity, and completeness and explored whether reviewers commented on the manuscript's importance, robustness, interpretation, discussion of results, and RCT reporting. Two investigators evaluated the review reports independently, with conflict resolution involving a third author.

Results: We sampled 26 RCTs and evaluated their 59 first peer review reports. Median word count was 276 (range=0-1047). Only 11 reports were constructive (19%), suggesting solutions for the problems noted. Of reviewers commenting on the manuscript's methods section (n=46/59,78%), 74% (n=34/46) addressed the suitability of the methodology. Fewer commented on the adequacy of conclusions (n=15/59; 25%) or the applicability of results (n=5/59; 9%), or whether study limitations had been acknowledged by authors (n=11/59; 18%). Only four (7%) commented on open research practices, including deviations from protocols, completeness of reporting, and sharing of data and materials.

Conclusions: Peer review reports of published RCTs in oncology were short, superficial, and rarely constructive. Although there is indication that reviewers commented on study methodology, little attention was paid to study conclusions, deviation from study protocols, completeness of reporting or data availability. Such review reports would be of limited value to authors for improving their trial study manuscripts, or to editors in deciding on manuscript publication.