A survey of how biology researchers assess credibility when serving on grant and hiring committees

We surveyed 485 biology researchers who have served on committees for grant review or hiring and promotion decisions, to understand how they assess the credibility of research outputs in these contexts. We found that assessment of credibility is very important to researchers serving in these committees but researchers are dissatisfied with their ability to judge credibility and often use inappropriate proxies, such as journal reputation and impact factor, to do this. Non-traditional research outputs associated with Open Science practices are particularly hard to assess, despite their potential to signal and support credibility assessments. Current policy may be overly focused on assessment of impact, and there are opportunities to provide better solutions to enable researchers to judge the credibility of research.