Peer review of prediction model studies in oncology needs improvement: A systematic review of open peer review reports from BMC journals

Objectives

To evaluate the completeness and quality of open peer review reports from BioMed Central (BMC) journals for regression-based clinical prediction model studies in oncology, focusing on adherence to methodological standards, reporting guidelines, and constructive feedback.

Methods

We searched for published prediction model studies in the field of oncology, which were published in BioMed Central journals in 2021. Data extraction used the Assessment of review Reports with a Checklist Available to eDItors and Authors (ARCADIA) checklist (13-item tool assessing review quality) with additional criteria (eg, word count, focus of comments on manuscript sections). Two investigators independently evaluated all open peer reviews, with conflicts resolved involving a third researcher. Descriptive statistics and narrative synthesis were applied.

Results

Peer reviews were brief (median: 243 words; range: 0–677), with 82.7% focusing on methods or results but rarely addressing limitations (<20%) or generalizability. No reviewers verified adherence to reporting guidelines (eg, TRIPOD); only one reviewer mentioned guideline use. Reviews prioritized superficial issues (67.3% focused on presentation) over methodological rigor (38.5% evaluated statistical methods). There are 19.2% suggested statistical revisions and <1% addressed protocol deviations or data availability.

Conclusion

Our findings show that peer reviews of prediction models lack depth, methodological scrutiny, and enforcement of reporting standards. This risks clinical harm from biased models and perpetuates research waste. Reforms are urgently needed, including implementing reporting guidelines (eg, TRIPOD+AI), mandatory reviewer training, and recognition of peer review as scholarly labor. Journals must prioritize methodological rigor in reviews to ensure reliable prediction models and safeguard patient care.