Research Evaluation最新文献_第5页

Describing the state of a research network: A mixed methods approach to network evaluation. 描述研究网络的状态：网络评估的混合方法。

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-10-28 eCollection Date: 2023-04-01 DOI: 10.1093/reseval/rvac034

James M Bowen, Mathieu Ouimet, Justin Lawarée, Joanna Bielecki, Ashley Rhéaume, Caylee Greenberg, Valeria E Rac

{"title":"Describing the state of a research network: A mixed methods approach to network evaluation.","authors":"James M Bowen, Mathieu Ouimet, Justin Lawarée, Joanna Bielecki, Ashley Rhéaume, Caylee Greenberg, Valeria E Rac","doi":"10.1093/reseval/rvac034","DOIUrl":"10.1093/reseval/rvac034","url":null,"abstract":"<p><p>Diabetes Action Canada Strategy for Patient-Oriented Research (SPOR) Network in Chronic Disease was formed in 2016 and is funded primarily through the Canadian Institutes of Health Research (CIHR). We propose a novel mixed-methods approach to a network evaluation integrating the State of Network Evaluation framework and the Canadian Academy of Health Sciences (CAHS) preferred framework and indicators. We measure key network themes of connectivity, health and results, and impact and return on investment associated with health research networks. Our methods consist of a longitudinal cross-sectional network survey of members and social network analysis to examine Network Connectivity and assess the frequency of interactions, the topics discussed during them, and how networking effectively facilitates interactions and collaboration among members. Network Health will be evaluated through semistructured interviews, a membership survey inquiring about satisfaction and experience with the Network, and a review of documentary sources related to funding and infrastructure to evaluate Network Sustainability. Finally, we will examine Network Results and Impact using the CAHS preferred framework and indicators to measure returns on investment in health research across the five domains of the CAHS framework, which include: advancing knowledge, capacity building, informing decision making, health impact, and economic and social impact. Indicators will be assessed with various methods, including bibliometric analyses, review of relevant documentary sources (annual reports), member activities informing health and research policy, and Patient Partner involvement. The Network Evaluation will provide members and stakeholders with information for planning, improvements, and funding future Network endeavors.</p>","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":"32 2","pages":"188-199"},"PeriodicalIF":3.3,"publicationDate":"2022-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10550251/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41173475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Correction to: Interdisciplinary research and policy impacts: Assessing the significance of knowledge coproduction 修正:跨学科研究和政策影响:评估知识合作生产的重要性

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-10-19 DOI: 10.1093/reseval/rvac039

引用次数: 1

Explaining employment sector choices of doctoral graduates in Germany 解读德国博士毕业生的就业选择

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-10-03 DOI: 10.1093/reseval/rvac030

Lea Goldan, Steffen Jaksztat, C. Gross

{"title":"Explaining employment sector choices of doctoral graduates in Germany","authors":"Lea Goldan, Steffen Jaksztat, C. Gross","doi":"10.1093/reseval/rvac030","DOIUrl":"https://doi.org/10.1093/reseval/rvac030","url":null,"abstract":"\u0000 Previous research in different national contexts has shown that individual preferences for certain job attributes, objective labour market conditions, subjective career prospects, and external encouragement shape doctoral graduates’ career decisions. For Germany, where the number of awarded doctoral degrees is highest within the European Union and where no established academic tenure-track system exists, the determinants of doctoral graduates’ sector choices are still largely unexplored. This article aims to shed light on the determinants of sector choices of doctoral graduates in Germany. By deriving the determinants from the wide version of rational choice theory and by measuring the determinants prior to employment sectors, we overcome the conceptual and methodological limitations of previous research. Using data from a nationally representative panel survey with doctoral graduates of the 2014 cohort in Germany, we differentiate between five distinct employment sectors and carry out multinomial logistic regression analysis. As expected, and in line with previous research from other countries, the results confirm that the sector choices of doctoral graduates in Germany depend on their preferences as well as various objective and subjective constraints. The article helps to better understand how doctoral graduates select into different employment sectors and thus provides important insights into postdoctoral career trajectories.","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":" ","pages":""},"PeriodicalIF":3.3,"publicationDate":"2022-10-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46824119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Research governance and the dynamics of science: A framework for the study of governance effects on research fields 研究治理与科学动态：研究治理对研究领域影响的框架

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-08-19 DOI: 10.1093/reseval/rvac028

M. Nedeva, Mayra M. Tirado, Duncan A. Thomas

{"title":"Research governance and the dynamics of science: A framework for the study of governance effects on research fields","authors":"M. Nedeva, Mayra M. Tirado, Duncan A. Thomas","doi":"10.1093/reseval/rvac028","DOIUrl":"https://doi.org/10.1093/reseval/rvac028","url":null,"abstract":"\u0000 This article offers a framework for the study of research governance effects on scientific fields framed by notions of research quality and the epistemic, organizational, and career choices they entail. The framework interprets the contested idea of ‘quality’ as an interplay involving notion origins, quality attributes, and contextual sites. We mobilize the origin and site components, to frame organizational-level events where quality notions inform selections, or selection events. Through the dynamic interplay between notions selected at specific sites, we contend, local actors enact research quality cumulatively, by making choices that privilege certain notions over others. In this article, we contribute in four ways. First, we propose an approach to study research governance effects on scientific fields. Second, we introduce first- and second-level effects of research governance paving the way to identify mechanisms through which these different levels of effects occur. Third, we assert that interactions between research spaces and fields leading to effects occur in the context of research organizations, and at nine key selection events. Fourth, and lastly, we discuss an empirical test on an illustration case to demonstrate how this approach can be applied.","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":" ","pages":""},"PeriodicalIF":3.3,"publicationDate":"2022-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45780294","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The impact of Italian performance-based research funding systems on the intensity of international research collaboration 意大利基于绩效的研究资助制度对国际研究合作强度的影响

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-07-29 DOI: 10.1093/reseval/rvac026

G. Abramo, Ciriaco Andrea D’Angelo

{"title":"The impact of Italian performance-based research funding systems on the intensity of international research collaboration","authors":"G. Abramo, Ciriaco Andrea D’Angelo","doi":"10.1093/reseval/rvac026","DOIUrl":"https://doi.org/10.1093/reseval/rvac026","url":null,"abstract":"\u0000 The study of national research assessment exercises serves to evaluate the effectiveness of policies versus their objectives and to improve the formulation of future initiatives. The aim of the current study is to verify whether the introduction of the first performance-based research funding in Italy, based on the 2004–10 VQR assessment, achieved the intended objective of inducing greater international collaboration on the part of researchers. For this, we apply a bibliometric approach based on the observation of coauthorships in Italian and worldwide scientific publications over a 14-year period, beginning in the target years of the VQR assessment. Through an Interrupted Time Series Analysis, we compare the expected and observed patterns of international coauthorship for Italy and the rest of the world. Although the rate of internationalization of Italian research is seen to be increasing, whether this is a consequence of the VQR incentives, or rather part of a global phenomenon of recourse to international collaboration in response to the increasingly complex scientific challenges, it is open to interpretation.","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":" ","pages":""},"PeriodicalIF":3.3,"publicationDate":"2022-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47425606","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Tilting at twin windmills: On article quotas and journal impact factors 在双风车上倾斜：论文章配额和期刊影响因素

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-07-28 DOI: 10.1093/reseval/rvac024

J. Serafy

引用次数: 0

Evaluation of the arts in performance-based research funding systems: An international perspective 基于绩效的研究资助体系中的艺术评估：国际视角

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-07-18 DOI: 10.1093/reseval/rvac017

Kamila Lewandowska, Emanuel Kulczycki, M. Ochsner

引用次数: 2

Quis judicabit ipsos judices? A case study on the dynamics of competitive funding panel evaluations 为什么是jusicabit ipsos jusices ?关于竞争性供资小组评价动态的个案研究

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-07-15 DOI: 10.1093/reseval/rvac021

João M. Santos

{"title":"Quis judicabit ipsos judices? A case study on the dynamics of competitive funding panel evaluations","authors":"João M. Santos","doi":"10.1093/reseval/rvac021","DOIUrl":"https://doi.org/10.1093/reseval/rvac021","url":null,"abstract":"\u0000 Securing research funding is essential for all researchers. The standard evaluation method for competitive grants is through evaluation by a panel of experts. However, the literature notes that peer review has inherent flaws and is subject to biases, which can arise from differing interpretations of the criteria, the impossibility for a group of reviewers to be experts in all possible topics within their field, and the role of affect. As such, understanding the dynamics at play during panel evaluations is crucial to allow researchers a better chance at securing funding, and also for the reviewers themselves to be aware of the cognitive mechanisms underlying their decision-making. In this study, we conduct a case study based on application and evaluation data for two social sciences panels in a competitive state-funded call in Portugal. Using a mixed-methods approach, we find that qualitative evaluations largely resonate with the evaluation criteria, and the candidate’s scientific output is partially aligned with the qualitative evaluations, but scientometric indicators alone do not significantly influence the candidate’s evaluation. However, the polarity of the qualitative evaluation has a positive influence on the candidate’s evaluation. This paradox is discussed as possibly resulting from the occurrence of a halo effect in the panel’s judgment of the candidates. By providing a multi-methods approach, this study aims to provide insights that can be useful for all stakeholders involved in competitive funding evaluations.","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":" ","pages":""},"PeriodicalIF":3.3,"publicationDate":"2022-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44940876","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Valorization of transdisciplinary research: An evaluation approach and empirical illustration 跨学科研究的价值化:一种评估方法与实证说明

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-06-30 DOI: 10.1093/reseval/rvac019

Stefania Munaretto, C. Mooren, L. Hessels

{"title":"Valorization of transdisciplinary research: An evaluation approach and empirical illustration","authors":"Stefania Munaretto, C. Mooren, L. Hessels","doi":"10.1093/reseval/rvac019","DOIUrl":"https://doi.org/10.1093/reseval/rvac019","url":null,"abstract":"\u0000 In recent times, there has been a surge of impact-oriented, transdisciplinary research programmes and projects integrating multiple disciplines, types of knowledge and practices. An essential element often mentioned in the literature to improve the performance of these programmes and support impact delivery is continuous reflection and learning via evaluation. We argue that because a standard format for organizing transdisciplinary research does not exist, tailor-made evaluation approaches designed around the specificities of each programme are needed. The existing evaluation literature provides useful building blocks that can be integrated and adapted to specific transdisciplinary research contexts. In this article, we develop the valorization cycle and apply it to evaluate a transdisciplinary research programme in the water sector. The building blocks of our approach are: understanding of valorization as a cyclical process; theory of change as a logic model to structure the evaluation; productive interactions as a process indicator articulated in learning outcomes (cognitive, relational, and strategic); and impact pathways as narratives to explain impact dynamics. Our framework is based both on research evaluation and learning literature and on our personal experience in the evaluation of transdisciplinary research. The evaluation of the programme showed how the valorization cycle can point to different learning outcomes across the research process that are conducive to impact, and provided useful insights to the programme managers to adjust the programme. The principle of tailoring an evaluation approach to the specificities of the programme evaluated using building blocks from the literature will be applicable in other transdisciplinary contexts too.","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":" ","pages":""},"PeriodicalIF":3.3,"publicationDate":"2022-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44778904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Under pressure: The extent and distribution of perceived pressure among scientists in Germany, Austria, and Switzerland 压力下：德国、奥地利和瑞士科学家感知压力的程度和分布

IF 3.3 4区管理学

Research Evaluation Pub Date : 2022-06-21 DOI: 10.1093/reseval/rvac014

David Johann, Isabel J. Raabe, H. Rauhut

{"title":"Under pressure: The extent and distribution of perceived pressure among scientists in Germany, Austria, and Switzerland","authors":"David Johann, Isabel J. Raabe, H. Rauhut","doi":"10.1093/reseval/rvac014","DOIUrl":"https://doi.org/10.1093/reseval/rvac014","url":null,"abstract":"\u0000 While it has been stressed repeatedly that academics nowadays have come to face extensive pressure, the extent and distribution of pressure to publish and to secure third-party funding has not been systematically investigated on a large scale. Based on the Zurich Survey of Academics, a representative large-scale web survey among academics working at universities in Germany, Austria, and Switzerland (DACH region), which measures perceived pressure using six-point Likert scales, this article examines the extent and distribution of pressure to publish and to attract external funding. Specifically, we examine differences in perceived pressure across countries, disciplines, types of higher education institutions, sociodemographic characteristics (academic status, age, gender, relationship status, number of children), and working conditions (non-tenured/tenured, time available for research). It is shown that researchers in the DACH region feel a high level of pressure, with the pressure to publish being slightly greater than the pressure to attract external funding. The results also suggest that perceived pressure is not evenly distributed among countries and groups of academics. Specifically, the results suggest that (1) more secure and permanent (tenured) positions should be created and (2) the high-performance culture should be addressed in at least some disciplines if excessive pressure is to be alleviated. The findings also suggest that further investments should be made (3) to promote equal career opportunities for women and men and (4) to ensure that scientists have sufficient time for their research and are not too occupied with other responsibilities, such as teaching or administrative duties.","PeriodicalId":47668,"journal":{"name":"Research Evaluation","volume":" ","pages":""},"PeriodicalIF":3.3,"publicationDate":"2022-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46993665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3