Quantitative Science Studies最新文献_第7页

How reliable are unsupervised author disambiguation algorithms in the assessment of research organization performance? 在评估研究机构绩效时，无监督作者消歧算法的可靠性如何?

IF 6.4

Quantitative Science Studies Pub Date : 2022-09-07 DOI: 10.1162/qss_a_00236

G. Abramo, Ciriaco Andrea D’Angelo

{"title":"How reliable are unsupervised author disambiguation algorithms in the assessment of research organization performance?","authors":"G. Abramo, Ciriaco Andrea D’Angelo","doi":"10.1162/qss_a_00236","DOIUrl":"https://doi.org/10.1162/qss_a_00236","url":null,"abstract":"Abstract Assessing the performance of universities by output to input indicators requires knowledge of the individual researchers working within them. Although in Italy the Ministry of University and Research updates a database of university professors, in all those countries where such databases are not available, measuring research performance is a formidable task. One possibility is to trace the research personnel of institutions indirectly through their publications, using bibliographic repertories together with author names disambiguation algorithms. This work evaluates the goodness-of-fit of the Caron and van Eck, CvE unsupervised algorithm by comparing the research performance of Italian universities resulting from its application for the derivation of the universities’ research staff, with that resulting from the supervised algorithm of D’Angelo, Giuffrida, and Abramo (2011), which avails of input data. Results show that the CvE algorithm overestimates the size of the research staff of organizations by 56%. Nonetheless, the performance scores and ranks recorded in the two compared modes show a significant and high correlation. Still, nine out of 69 universities show rank deviations of two quartiles. Measuring the extent of distortions inherent in any evaluation exercises using unsupervised algorithms, can inform policymakers’ decisions on building national research staff databases, instead of settling for the unsupervised approaches.","PeriodicalId":34021,"journal":{"name":"Quantitative Science Studies","volume":"4 1","pages":"144-166"},"PeriodicalIF":6.4,"publicationDate":"2022-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45627516","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

A converging global research system 一个融合的全球研究体系

IF 6.4

Quantitative Science Studies Pub Date : 2022-09-06 DOI: 10.1162/qss_a_00208

Jonathan Adams, M. Szomszor

{"title":"A converging global research system","authors":"Jonathan Adams, M. Szomszor","doi":"10.1162/qss_a_00208","DOIUrl":"https://doi.org/10.1162/qss_a_00208","url":null,"abstract":"Abstract We examine the hypothesis that research collaboration has enabled a global research network to evolve, with self-organizing properties transcending national research policy. We examine research output, bilateral and multilateral collaboration, subject diversity, and citation impact over 40 years, in detail for the G7 and BRICK groups of countries and in summary for 26 other nations. We find that the rise in national output was strongly associated with bilateral collaboration until the 2000s but after that by multilateral partnerships, with the shift happening at much the same time across countries. There was a general increase in research subject diversity, with evenness across subjects converging on a similar index value for many countries. Similar diversity is not the same as actual similarity but, in fact, the G7 countries became increasingly similar. National average citation impact (CNCI) rose and groups converged on similar impact values. The impact of the largest economies is above world average, which is a phenomenon we discuss separately. The similarities in patterns and timing occur across countries despite variance in their research policies, such as research assessment. We suggest that the key agent facilitating global network self-organization is a shared concept of best practice in research.","PeriodicalId":34021,"journal":{"name":"Quantitative Science Studies","volume":"3 1","pages":"715-731"},"PeriodicalIF":6.4,"publicationDate":"2022-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44132027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Recategorising research: Mapping from FoR 2008 to FoR 2020 in Dimensions 重新分类研究：从2008年系列到2020年系列的维度映射

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-31 DOI: 10.1162/qss_a_00244

Simon Porter, Daniel W. Hook

引用次数: 3

Peer reviewer topic choice and its impact on interrater reliability: A mixed-method study 同行评议主题选择及其对评议可信度的影响:一项混合方法研究

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-25 DOI: 10.1162/qss_a_00207

Thomas Feliciani, Junwen Luo, K. Shankar

{"title":"Peer reviewer topic choice and its impact on interrater reliability: A mixed-method study","authors":"Thomas Feliciani, Junwen Luo, K. Shankar","doi":"10.1162/qss_a_00207","DOIUrl":"https://doi.org/10.1162/qss_a_00207","url":null,"abstract":"Abstract One of the main critiques of academic peer review is that interrater reliability (IRR) among reviewers is low. We examine an underinvestigated factor possibly contributing to low IRR: reviewers’ diversity in their topic-criteria mapping (“TC-mapping”). It refers to differences among reviewers pertaining to which topics they choose to emphasize in their evaluations, and how they map those topics onto various evaluation criteria. In this paper we look at the review process of grant proposals in one funding agency to ask: How much do reviewers differ in TC-mapping, and do their differences contribute to low IRR? Through a content analysis of review forms submitted to a national funding agency (Science Foundation Ireland) and a survey of its reviewers, we find evidence of interreviewer differences in their TC-mapping. Using a simulation experiment we show that, under a wide range of conditions, even strong differences in TC-mapping have only a negligible impact on IRR. Although further empirical work is needed to corroborate simulation results, these tentatively suggest that reviewers’ heterogeneous TC-mappings might not be of concern for designers of peer review panels to safeguard IRR.","PeriodicalId":34021,"journal":{"name":"Quantitative Science Studies","volume":"3 1","pages":"832-856"},"PeriodicalIF":6.4,"publicationDate":"2022-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45723631","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An open data set of scholars on Twitter 推特上学者的开放数据集

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-23 DOI: 10.1162/qss_a_00250

P. Mongeon, T. Bowman, R. Costas

引用次数: 5

Visualizing academic descendants using modified Pavlo diagrams: Results based on five researchers in biomechanics and biomedicine 使用修改的Pavlo图可视化学术后代:基于生物力学和生物医学五位研究人员的结果

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-20 DOI: 10.1162/qss_a_00205

W. Lievers

{"title":"Visualizing academic descendants using modified Pavlo diagrams: Results based on five researchers in biomechanics and biomedicine","authors":"W. Lievers","doi":"10.1162/qss_a_00205","DOIUrl":"https://doi.org/10.1162/qss_a_00205","url":null,"abstract":"Abstract Visualizing the academic descendants of prolific researchers is a challenging problem. To this end, a modified Pavlo algorithm is presented and its utility is demonstrated based on manually collected academic genealogies of five researchers in biomechanics and biomedicine. The researchers have 15–32 children each and between 93 and 384 total descendants. The graphs generated by the modified algorithm were over 97% smaller than the original. Mentorship metrics were also calculated; their hm-indices are 5–7 and the gm-indices are in the range 7–13. Of the 1,096 unique researchers across the five family trees, 153 (14%) had graduated their own PhD students by the end of 2021. It took an average of 9.6 years after their own graduation for an advisor to graduate their first PhD student, which suggests that an academic generation in this field is approximately one decade. The manually collected data sets used were also compared against the crowd-sourced academic genealogy data from the AcademicTree.org website. The latter included only 45% of the people and 34% of the connections, so this limitation must be considered when using it for analyses where completeness is required. The data sets and an implementation of the algorithm are available for reuse.","PeriodicalId":34021,"journal":{"name":"Quantitative Science Studies","volume":"3 1","pages":"489-511"},"PeriodicalIF":6.4,"publicationDate":"2022-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64426870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Measuring and interpreting the differences of the nations’ scientific specialization indexes by output and by input 以产出和投入衡量和解释各国科学专业化指标的差异

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-20 DOI: 10.1162/qss_a_00206

G. Abramo, Ciriaco Andrea D’Angelo, F. D. Costa

引用次数: 1

Which factors are associated with Open Access publishing? A Springer Nature case study 哪些因素与开放获取出版有关?b施普林格自然案例研究

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-17 DOI: 10.1162/qss_a_00253

Fakhri Momeni, S. Dietze, Philipp Mayr, Kristin Biesenbender, Isabella Peters

{"title":"Which factors are associated with Open Access publishing? A Springer Nature case study","authors":"Fakhri Momeni, S. Dietze, Philipp Mayr, Kristin Biesenbender, Isabella Peters","doi":"10.1162/qss_a_00253","DOIUrl":"https://doi.org/10.1162/qss_a_00253","url":null,"abstract":"Abstract Open Access (OA) facilitates access to research articles. However, authors or funders often must pay the publishing costs, preventing authors who do not receive financial support from participating in OA publishing and gaining citation advantage for OA articles. OA may exacerbate existing inequalities in the publication system rather than overcome them. To investigate this, we studied 522,411 articles published by Springer Nature. Employing correlation and regression analyses, we describe the relationship between authors affiliated with countries from different income levels, their choice of publishing model, and the citation impact of their papers. A machine learning classification method helped us to explore the importance of different features in predicting the publishing model. The results show that authors eligible for article processing charge (APC) waivers publish more in gold OA journals than others. In contrast, authors eligible for an APC discount have the lowest ratio of OA publications, leading to the assumption that this discount insufficiently motivates authors to publish in gold OA journals. We found a strong correlation between the journal rank and the publishing model in gold OA journals, whereas the OA option is mostly avoided in hybrid journals. Also, results show that the countries’ income level, seniority, and experience with OA publications are the most predictive factors for OA publishing in hybrid journals.","PeriodicalId":34021,"journal":{"name":"Quantitative Science Studies","volume":"4 1","pages":"353-371"},"PeriodicalIF":6.4,"publicationDate":"2022-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48857752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

An improved practical approach to forecasting exceptional growth in research 一种改进的预测研究异常增长的实用方法

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-16 DOI: 10.1162/qss_a_00202

K. Boyack, R. Klavans

引用次数: 1

AOC: Assembling overlapping communities AOC:组装重叠的社区

IF 6.4

Quantitative Science Studies Pub Date : 2022-08-05 DOI: 10.1162/qss_a_00227

Akhil Jakatdar, T. Warnow, George Chacko

引用次数: 2