{"title":"Distinguishing articles in questionable and non-questionable journals using quantitative indicators associated with quality","authors":"D. Stephen","doi":"10.55835/644245cb8e703ddb4dc07eda","DOIUrl":"https://doi.org/10.55835/644245cb8e703ddb4dc07eda","url":null,"abstract":"This ongoing study investigates the viability of distinguishing articles in questionable journals (QJs) from those in non-QJs on the basis of quantitative indicators typically associated with quality, and what can be deduced about the quality of articles in QJs based on the differences observed. I contrast the length of abstracts and full-texts, prevalence of spelling errors, text readability, number of references and citations, and other characteristics of 1,714 articles from 31 QJs, 1,691 articles from 16 journals indexed in Web of Science (WoS), and 1,900 articles from 45 non-WoS/non-QJs, all in the field of psychology. Initial results indicate that there are differences between QJs and non-QJ samples, however these are relatively small, perhaps indicating that QJs may not substantially differ from non-QJs on these quantitative indicators of quality. However, I intend to use additional analyses to further explore any potential differences.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121444075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Measuring disruptiveness and continuity of research by using the Disruption Index (DI) – A Bayesian statistical approach","authors":"Rüdiger Mutz, L. Bornmann","doi":"10.55835/644117475a1411a1cb49918d","DOIUrl":"https://doi.org/10.55835/644117475a1411a1cb49918d","url":null,"abstract":"L. Wu, Wang, and Evans (2019) introduced the disruption index (DI) which has been designed to capture disruptiveness of individual publications based on dynamic citation networks of publications. In this study, we propose a statistical modelling approach to tackle open questions with the DI: (1) how to consider uncertainty in the calculation of DI values, (2) how to aggregate DI values for paper sets, (3) how to predict DI values using covariates, and (4) how to unambiguously classify papers into either disruptive or not disruptive. A Bayesian multilevel logistic approach is suggested that extends an approach of Figueiredo and Andrade (2019). A reanalysis of sample data from Bornmann and Tekles (2021) and Bittmann, Tekles, and Bornmann (2022) shows that the Bayesian approach is helpful in tackling the open questions. For example, the modelling approach is able to predict disruptive papers (milestone papers in physics) in a good way.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125079105","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How researchers collaborate across disciplines? Patterns of interdisciplinary collaboration based on a dual-perspective framework","authors":"Zhe Cao, Lin Zhang, Ying Huang","doi":"10.55835/644274f66972d54225e8e5f9","DOIUrl":"https://doi.org/10.55835/644274f66972d54225e8e5f9","url":null,"abstract":"With a combination of new data sources and mixed methods including bibliometrics, machine learning and network analysis, this study puts forward a new framework of categorizing different interdisciplinary collaboration patterns from a discipline-contribution perspective. Based on 20,542 research articles published on PLoS series of journals in 2018, 14,744 articles with interdisciplinary collaborations (ICAs) are recognized. By establishing six indicators that measure the variety, similarity and balance of authors’ disciplines and their contribution roles, ICAs are divided into four categories after the agglomerative hierarchical clustering. With a fine-grained analysis of the structural and correlation characteristics of authors’ disciplines and contribution in different clusters, four interdisciplinary collaboration patterns of sheep flock, bee colony, intercropping and rainforest are found. Our results may contribute to developing new methodologies and theories of interdisciplinary collaboration, and enrich the understanding of interdisciplinary collaboration as well as relevant policies.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131372815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Can tweets predict article retractions?","authors":"Erdong Zheng, Hui-Zhen Fu, Zhichao Fang","doi":"10.55835/644126a8763e8d2091a0cfdc","DOIUrl":"https://doi.org/10.55835/644126a8763e8d2091a0cfdc","url":null,"abstract":"This study explores the potential of using tweets to predict article retractions, by analyzing the Twitter mention data of retracted articles as the treatment group and unretracted articles that were matched as a control group. The results show that tweets could predict article retractions with an accuracy of 57%-60% by machine learning models. Sentiment analysis is not effective in predicting article retractions. The study sheds light on a novel method of detecting scientific misconduct in the early stage.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"108 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131934931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Geographical distribution of high-novelty research","authors":"Kuniko Matsumoto","doi":"10.55835/643b16770dbbd7f8a6d7c898","DOIUrl":"https://doi.org/10.55835/643b16770dbbd7f8a6d7c898","url":null,"abstract":"In this study, trial analyses using bibliometric approaches were performed to investigate the geographical distribution of high-novelty research. Data on approximately 2.55 million academic papers published in 2021 were examined as a pilot to show worldwide statistical data on novelty research. A combinatorial novelty indicator measuring units comprising paired reference papers was adopted in the analyses. This study shows the main three results: the top 20 countries/regions in the top 10% of high-novelty papers, the share of the top 10% high-novelty papers in each country/region, and the share of the top 10% high-novelty papers by field in China and the United States, which contribute globally to the top 10% of high-novelty papers.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134423119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bianca Savegnago de Mira, D. Martínez-Ávila, Antonio Perianes-Rodríguez
{"title":"Beyond the Journal Impact Factor: a new approach to characterize the visibility of journals and articles based on percentiles","authors":"Bianca Savegnago de Mira, D. Martínez-Ávila, Antonio Perianes-Rodríguez","doi":"10.55835/6442b41bf6484429f00ec8f1","DOIUrl":"https://doi.org/10.55835/6442b41bf6484429f00ec8f1","url":null,"abstract":"The use of the Journal Impact Factor as an indicator of the quality of individual articles and journals presents limitations. Here, we propose a new approach that combines citation distributions by percentiles, the number and proportion of uncited articles, and the number and proportion of publications below the mean citations in relation to the journal and the field as a method to measure the influence of the publications and their real visibility. We tested it with 10 journals under the Information Science and Library Science category (LIS) of the JCR whose articles were retrieved from OpenAlex for the period 2000-2020 and the results suggest that journals whose graphs present a lower proportion of articles below the mean citation in the category, a lower proportion of uncited documents, and a greater area below the diagonal are those that have more visibility, regardless of the number of publications and their age.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124389055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Do popular research topics attract the most social attention? A first proposal based on OpenAlex and Wikipedia","authors":"Wenceslao Arroyo-Machado, R. Costas","doi":"10.55835/6442bb04903ef57acd6dab9e","DOIUrl":"https://doi.org/10.55835/6442bb04903ef57acd6dab9e","url":null,"abstract":"Altmetric research has seen its horizons expanded to the heterogeneity of interactions produced between scientific and non-scientific entities. In this context, Wikipedia stands out as a social media of particular interest as the page views of its articles have proven to be a valuable metric of social attention. The aim of this paper is to contribute to this new research stream by analysing whether the research topics of greatest academic interest align with those that attract the most social attention. To this end, the OpenAlex concepts are explored by comparing their works count with the page views of their respective Wikipedia articles. As a result, a correlation analysis between the two metrics reveals a lack of connection between the two realms. Likewise, root-level concepts are explored to illustrate such a difference.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131594989","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jodi Schneider, Jounghyoun Lee, Heng Zheng, Malik Oyewale Salami
{"title":"Assessing the agreement in retraction indexing across 4 multidisciplinary sources: Crossref, Retraction Watch, Scopus, and Web of Science","authors":"Jodi Schneider, Jounghyoun Lee, Heng Zheng, Malik Oyewale Salami","doi":"10.55835/6441e5cae04dbe5586d06a5f","DOIUrl":"https://doi.org/10.55835/6441e5cae04dbe5586d06a5f","url":null,"abstract":"Previous research has posited a correlation between poor indexing and inadvertent post-retraction citation. However, to date, there has been limited systematic study of retraction indexing quality: we are aware of one database-wide comparison of PubMed and Web of Science, and multiple smaller studies highlighting indexing problems for items with the same reason for retraction or same field of study. To assess the agreement between multidisciplinary retraction indexes, we create a union list of 49,924 publications with DOIs from the retraction indices of at least one of Crossref, Retraction Watch, Scopus, and Web of Science. Only 1593 (3%) are deemed retracted by the intersection of all four sources. For 14,743 publications (almost 30%), there is disagreement: at least one source deems them retracted while another lacks retraction indexing. Of the items deemed retracted by at least one source, retraction indexing was lacking for 32% covered in Scopus, 7% covered in Crossref, and 4% covered in Web of Science. We manually examined 201 items from the union list and found that 115/201 (57.21%) DOIs were retracted publications while 59 (29.35%) were retraction notices. In future work we plan to use a validated version of this union list to assess the retraction indexing of subject-specific sources.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132505410","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Country shifts in the authorship of conference papers","authors":"A. Guskov, Denis Kosyakov","doi":"10.55835/643fadb94e97d59d99bef125","DOIUrl":"https://doi.org/10.55835/643fadb94e97d59d99bef125","url":null,"abstract":"There are significant scatters in indexed conference papers quantity in different fields of science and countries that are of a different nature. This paper proposes a methodology for calculating disciplinary and institutional shifts of publishing in conference proceedings and its application to country analysis based on Scopus data. A disciplinary shift is the deviation of the conference paper share of a country from the world average, which is caused by the scientific specialization of this country. The institutional shift is the deviation of the conference paper share of a country from the world average, caused by the national specifics of the science policies, in particular, excessive stimulation of publication activity. The study confirms previously observed institutional shifts in the Czech Republic, Russia, and Indonesia, and identifies several more countries where there may also be distortions, caused by science policies.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114838293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Do female academics submit fewer grant applications than men?","authors":"Torger Möller","doi":"10.55835/644303d4b2b5580ba561581a","DOIUrl":"https://doi.org/10.55835/644303d4b2b5580ba561581a","url":null,"abstract":"The state of the research on grant application behaviour is that female academics submit fewer proposals than men. This study points out that it is methodologically challenging to draw this conclusion. We know a lot about applicants, but little about the pool of potential applicants as the underlying population. We use a random sample of academics as potential applicants to investigate the grant application activity of male and female researchers. The results show that when an appropriate benchmark is applied (in this case, controlling for academic status and research area), no significant gender differences in grant applications can be found.","PeriodicalId":334841,"journal":{"name":"27th International Conference on Science, Technology and Innovation Indicators (STI 2023)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130563033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}