Community-driven reviewing and validation of publications

Abstract

In this report, we share our practical experience on crowdsourcing evaluation of research artifacts and reviewing of publications since 2008. We also briey discuss encountered problems including reproducibility of experimental results and possible solutions.