Simulation-based evaluation of a crowdsourced expert peer review system

The primary goal of this paper is to propose and simulate crowdsourcing-based solutions which might optimize the scientific peer review system. More specifically, a global reviewer database and gamification techniques will be proposed with the goal of obtaining more high-quality reviews for papers received by journals. The proposed modifications were assessed in a multi-agent simulation environment, in which the members of the reviewer crowd were modeled as agents. Our simulation-based evaluations implemented in the MASON multi-agent environment showed that the introduction of the above improvements would allow editors to find the most suitable and responsive reviewers, as well as to lower the number of scientific papers which do not receive enough reviews.