Can AI help authors prepare better risk science manuscripts?

Abstract

Scientists, publishers, and journal editors are wondering how, whether, and to what extent artificial intelligence (AI) tools might soon help to advance the rigor, efficiency, and value of scientific peer review. Will AI provide timely, useful feedback that helps authors improve their manuscripts while avoiding the biases and inconsistencies of human reviewers? Or might it instead generate low-quality verbiage, add noise and errors, reinforce flawed reasoning, and erode trust in the review process? This perspective reports on evaluations of two experimental AI systems: (i) a "Screener" available at http://screener.riskanalysis.cloud/ that gives authors feedback on whether a draft paper (or abstract, proposal, etc.) appears to be a fit for the journal Risk Analysis, based on the guidance to authors provided by the journal (https://www.sra.org/journal/what-makes-a-good-risk-analysis-article/); and (ii) a more ambitious "Reviewer" (http://aia1.moirai-solutions.com/) that gives substantive technical feedback and recommends how to improve the clarity of methodology and the interpretation of results. The evaluations were conducted by a convenience sample of Risk Analysis Area Editors (AEs) and authors, including two authors of manuscripts in progress and four authors of papers that had already been published. The Screener was generally rated as useful. It has been deployed at Risk Analysis since January of 2025. On the other hand, the Reviewer had mixed ratings, ranging from strongly positive to strongly negative. This perspective describes both the lessons learned and potential next steps in making AI tools useful to authors prior to peer review by human experts.