Normalization and Valuation of Research Evaluation Indicators in Different Scientific Fields

Q3 Social Sciences

Journal of Information Science Theory and Practice Pub Date : 2016-03-30 DOI:10.1633/JISTAP.2016.4.1.2

A. N. Chakoli, Roghayeh Ghazavi

{"title":"Normalization and Valuation of Research Evaluation Indicators in Different Scientific Fields","authors":"A. N. Chakoli, Roghayeh Ghazavi","doi":"10.1633/JISTAP.2016.4.1.2","DOIUrl":null,"url":null,"abstract":"Given the difference in research performance in various scientific fields, this study aims to weight and valuate current indicators used for evaluation of scientific productions (publications), in order to adjust these indicators in comparison to each other and make possible a more precise evaluation of scientific productions. This is a scientometrics study using documentary, evaluative, and survey techniques. The statistical population consisted of 106 top Iranian researchers, scientists, and scientific and research managers. Then their research resume information was gathered and analyzed based on research questions. In order to compare values, the data gathered from research production performance of the population was weighted using Shannon entropy method. Also, the weights of each scientific production importance according to expert opinions (extracted from other works) was analyzed and after adjustment the final weight of each scientific production was determined. A pairwise matrix was used in order to determine the ratios. According to the results, in the area of engineering sciences, patents (0.142) in the area of science, international articles (0.074) in the area of humanities and social sciences, books (0.174), and in the area of medical sciences, international articles (0.111) had the highest weight compared to other information formats. By dividing the weights for each type of publication, the value of each scientific production compared to other scientific productions in the same field and productions of other fields was calculated. Validation of the results in the studied population resulted in very high credibility for all investigated indicators in all four fields. By using these values and normalized ratios of publication indicators it is possible to achieve precise and adjusted results, making it possible to feasibly use these results in realistic policy making.","PeriodicalId":37582,"journal":{"name":"Journal of Information Science Theory and Practice","volume":"22 6 1","pages":"21-29"},"PeriodicalIF":0.0000,"publicationDate":"2016-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Information Science Theory and Practice","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1633/JISTAP.2016.4.1.2","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Social Sciences","Score":null,"Total":0}

引用次数: 1

Abstract

Given the difference in research performance in various scientific fields, this study aims to weight and valuate current indicators used for evaluation of scientific productions (publications), in order to adjust these indicators in comparison to each other and make possible a more precise evaluation of scientific productions. This is a scientometrics study using documentary, evaluative, and survey techniques. The statistical population consisted of 106 top Iranian researchers, scientists, and scientific and research managers. Then their research resume information was gathered and analyzed based on research questions. In order to compare values, the data gathered from research production performance of the population was weighted using Shannon entropy method. Also, the weights of each scientific production importance according to expert opinions (extracted from other works) was analyzed and after adjustment the final weight of each scientific production was determined. A pairwise matrix was used in order to determine the ratios. According to the results, in the area of engineering sciences, patents (0.142) in the area of science, international articles (0.074) in the area of humanities and social sciences, books (0.174), and in the area of medical sciences, international articles (0.111) had the highest weight compared to other information formats. By dividing the weights for each type of publication, the value of each scientific production compared to other scientific productions in the same field and productions of other fields was calculated. Validation of the results in the studied population resulted in very high credibility for all investigated indicators in all four fields. By using these values and normalized ratios of publication indicators it is possible to achieve precise and adjusted results, making it possible to feasibly use these results in realistic policy making.

查看原文本刊更多论文

不同科学领域研究评价指标的规范化与评价

鉴于各个科学领域的研究绩效存在差异，本研究旨在对目前用于科学成果(出版物)评价的指标进行加权和评价，以便在相互比较中调整这些指标，从而更准确地评价科学成果。这是一项科学计量学研究，使用文献、评估和调查技术。统计人口包括106名伊朗顶级研究人员、科学家和科研管理人员。然后收集他们的研究简历信息，并根据研究问题进行分析。为了进行数值比较，采用香农熵法对种群的研究生产绩效数据进行加权。并根据专家意见(摘自其他著作)对各科学成果重要性权重进行分析，调整后确定各科学成果的最终权重。为了确定比率，使用了一对矩阵。结果显示，与其他信息格式相比，工程科学领域的专利(0.142)，人文社会科学领域的国际文章(0.074)，书籍(0.174)，医学科学领域的国际文章(0.111)的权重最高。通过除以每种出版物的权重，计算出每种科学成果与同一领域的其他科学成果和其他领域的成果相比的价值。在研究人群中验证结果导致所有四个领域的所有调查指标具有非常高的可信度。通过使用这些数值和出版物指标的标准化比率，可以获得精确和调整的结果，从而可以在现实的政策制定中可行地使用这些结果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Information Science Theory and Practice Social Sciences-Library and Information Sciences

CiteScore

1.10

自引率

0.00%

发文量

审稿时长

12 weeks

期刊介绍： The Journal of Information Science Theory and Practice (JISTaP) is an international journal that aims at publishing original studies, review papers and brief communications on information science theory and practice. The journal provides an international forum for practical as well as theoretical research in the interdisciplinary areas of information science, such as information processing and management, knowledge organization, scholarly communication and bibliometrics. To foster scholarly communication among researchers and practitioners of library and information science around the globe, JISTaP offers a no-fee open access publishing venue where a team of dedicated editors, reviewers and staff members volunteer their services to ensure rapid dissemination and communication of scholarly works that make significant contributions. In a modern society, where information production and consumption grow at an astronomical rate, the science of information management, organization, and analysis is invaluable in effective utilization of information. The key objective of the journal is to foster research that can contribute to advancements and innovations in the theory and practice of information and library science so as to promote timely application of the findings from scientific investigations to everyday life. Recognizing the importance of the global perspective with understanding of region-specific issues, JISTaP encourages submissions of manuscripts that discuss global implications of regional findings as well as regional implications of global findings.