Targeted, actionable and fair: Reviewer reports as feedback and its effect on ECR career choices

IF 2.9 4区管理学 Q1 INFORMATION SCIENCE & LIBRARY SCIENCE

Research Evaluation Pub Date : 2023-11-02 DOI:10.1093/reseval/rvad034

Gemma Elizabeth Derrick, Alessandra Zimmermann, Helen Greaves, Jonathan Best, Richard Klavans

{"title":"Targeted, actionable and fair: Reviewer reports as feedback and its effect on ECR career choices","authors":"Gemma Elizabeth Derrick, Alessandra Zimmermann, Helen Greaves, Jonathan Best, Richard Klavans","doi":"10.1093/reseval/rvad034","DOIUrl":null,"url":null,"abstract":"Abstract Previous studies of the use of peer review for the allocation of competitive funding agencies have concentrated on questions of efficiency and how to make the ‘best’ decision, by ensuring that successful applicants are also the more productive or visible in the long term. This paper examines the components of feedback received from an unsuccessful grant application, is associated with motivating applicants career decisions to persist (reapply for funding at T1), or to switch (not to reapply, or else leave academia). This study combined data from interviews with unsuccessful ECR applicants (n = 19) to The Wellcome Trust 2009–19, and manual coding of reviewer comments received by applicants (n = 81). All applicants received feedback on their application at T0 with a large proportion of unsuccessful applicants reapplying for funding at T1. Here, peer-review-comments-as-feedback sends signals to applicants to encourage them to persist (continue) or switch (not continue) even when the initial application has failed. Feedback associated by unsuccessful applicants as motivating their decision to resubmit had three characteristics: actionable; targeted; and fair. The results lead to identification of standards of feedback for funding agencies and peer-reviewers to promote when providing reviewer feedback to applicants as part of their peer review process. The provision of quality reviewer-reports-as-feedback to applicants, ensures that peer review acts as a participatory research governance tool focused on supporting the development of individuals and their future research plans.","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":"13 3","pages":"0"},"PeriodicalIF":2.9000,"publicationDate":"2023-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Research Evaluation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1093/reseval/rvad034","RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"INFORMATION SCIENCE & LIBRARY SCIENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Abstract Previous studies of the use of peer review for the allocation of competitive funding agencies have concentrated on questions of efficiency and how to make the ‘best’ decision, by ensuring that successful applicants are also the more productive or visible in the long term. This paper examines the components of feedback received from an unsuccessful grant application, is associated with motivating applicants career decisions to persist (reapply for funding at T1), or to switch (not to reapply, or else leave academia). This study combined data from interviews with unsuccessful ECR applicants (n = 19) to The Wellcome Trust 2009–19, and manual coding of reviewer comments received by applicants (n = 81). All applicants received feedback on their application at T0 with a large proportion of unsuccessful applicants reapplying for funding at T1. Here, peer-review-comments-as-feedback sends signals to applicants to encourage them to persist (continue) or switch (not continue) even when the initial application has failed. Feedback associated by unsuccessful applicants as motivating their decision to resubmit had three characteristics: actionable; targeted; and fair. The results lead to identification of standards of feedback for funding agencies and peer-reviewers to promote when providing reviewer feedback to applicants as part of their peer review process. The provision of quality reviewer-reports-as-feedback to applicants, ensures that peer review acts as a participatory research governance tool focused on supporting the development of individuals and their future research plans.

查看原文本刊更多论文

目标明确，可操作和公平:审稿人报告作为反馈及其对ECR职业选择的影响

以前关于在竞争性资助机构分配中使用同行评议的研究集中在效率问题和如何做出“最佳”决策上，通过确保成功的申请人在长期内也更富有成效或更引人注目。本文研究了从一个不成功的资助申请中收到的反馈的组成部分，这些反馈与激励申请人的职业决定有关，是坚持(在T1重新申请资助)，还是转换(不重新申请，或者离开学术界)。本研究结合了2009-19年惠康信托基金未成功ECR申请人的访谈数据(n = 19)，以及申请人收到的审稿人意见的手工编码(n = 81)。所有申请人均于大学一年级时收到有关申请的反馈，而大部分落选申请人在大学一年级时重新申请资助。在这里，同行评议即反馈向申请人发出信号，鼓励他们坚持(继续)或切换(不继续)，即使最初的申请失败了。不成功申请者的反馈能够激励他们重新提交申请，这些反馈有三个特点:可操作;有针对性;和公平的。研究结果确定了资助机构和同行审稿人在向申请人提供审稿人反馈作为同行评审过程的一部分时应促进的反馈标准。向申请人提供高质量的审稿人报告作为反馈，确保同行评议作为一种参与性研究治理工具，重点是支持个人的发展及其未来的研究计划。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Research Evaluation INFORMATION SCIENCE & LIBRARY SCIENCE-

CiteScore

6.00

自引率

18.20%

发文量

期刊介绍： Research Evaluation is a peer-reviewed, international journal. It ranges from the individual research project up to inter-country comparisons of research performance. Research projects, researchers, research centres, and the types of research output are all relevant. It includes public and private sectors, natural and social sciences. The term "evaluation" applies to all stages from priorities and proposals, through the monitoring of on-going projects and programmes, to the use of the results of research.