{"title":"Enhancing peer review efficiency: A mixed-methods analysis of artificial intelligence-assisted reviewer selection across academic disciplines","authors":"Shai Farber","doi":"10.1002/leap.1638","DOIUrl":"https://doi.org/10.1002/leap.1638","url":null,"abstract":"<p>This mixed-methods study evaluates the efficacy of artificial intelligence (AI)-assisted reviewer selection in academic publishing across diverse disciplines. Twenty journal editors assessed AI-generated reviewer recommendations for a manuscript. The AI system achieved a 42% overlap with editors' selections and demonstrated a significant improvement in time efficiency, reducing selection time by 73%. Editors found that 37% of AI-suggested reviewers who were not part of their initial selection were indeed suitable. The system's performance varied across disciplines, with higher accuracy in STEM fields (Cohen's <i>d</i> = 0.68). Qualitative feedback revealed an appreciation for the AI's ability to identify lesser-known experts but concerns about its grasp of interdisciplinary work. Ethical considerations, including potential algorithmic bias and privacy issues, were highlighted. The study concludes that while AI shows promise in enhancing reviewer selection efficiency and broadening the reviewer pool, it requires human oversight to address limitations in understanding nuanced disciplinary contexts. Future research should focus on larger-scale longitudinal studies and developing ethical frameworks for AI integration in peer-review processes.</p>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1638","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524710","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploring the role of rejection in scholarly knowledge production: Insights from granular interaction thinking and information theory","authors":"Quan-Hoang Vuong, Minh-Hoang Nguyen","doi":"10.1002/leap.1636","DOIUrl":"https://doi.org/10.1002/leap.1636","url":null,"abstract":"<p>Rejection is an inevitable challenge that all scholars face when they enter academia. The rejections encountered during editorial evaluations and peer-review processes are typically seen as a filtering mechanism that helps distinguish between perceived qualified and perceived ineligible scientific works. Scientific works perceived as useful and reliable will proceed to publication, and those deemed ineligible will be excluded. Here, ‘perceived useful and reliable scientific works’ and ‘perceived ineligible works’ highlight the subjectivity inherent in evaluation processes driven by editors and reviewers. In this article, we aim to elaborate on the advantages and limitations of the rejection process through the lens of Shannon's information theory (Shannon, <span>1948</span>) and the theory of granular interaction thinking (Vuong & Nguyen, <span>2024a</span>), which is based on the worldviews of quantum mechanics and the mindsponge theory (Hertog, <span>2023</span>; Rovelli, <span>2018</span>; Susskind & Friedman, <span>2014</span>; Vuong, <span>2023</span>).</p><p>Given the finite information each person can process, knowledge production appears to be a dynamic, multi-state process requiring contributions from many individuals. Knowledge generated in former states (demonstrated by State 1) can be used as resources for knowledge production in subsequent states (demonstrated by State 2). In other words, knowledge is produced through the interactions between new observations, theoretical formulations, and useful knowledge accumulated in previous states of knowledge production. For instance, reaching the current stage of utilizing solar energy (which accounts for only 4.5% of total global electricity generation) has involved contributions of knowledge from myriad societies (e.g., Ancient Egypt, Ancient Greece, Ancient Rome) and great individuals (e.g., Archimedes, Mikhail Vasilyevich Lomonosov, Edmond Becquerel, Heinrich Hertz, Albert Einstein) over the course of 28 centuries (Petrova-Koch, <span>2020</span>).</p><p>As the future is probabilistically determined by the past but not unequivocally, maximizing the probability that useful knowledge (or scientific works) can be transmitted from State 1 to State 2 is crucial for upholding the effectiveness of the knowledge production process. Journal and book publishing aim to store and disseminate perceived useful scientific works in State 1 for potential reuse in State 2, thereby facilitating new knowledge generation. Although the rejection process imposes a mental burden on scholars, it is essential for achieving this goal.</p><p><span></span><math>\u0000 <mrow>\u0000 <mi>P</mi>\u0000 <mfenced>\u0000 <msub>\u0000 <mi>x</mi>\u0000 <mi>i</mi>\u0000 </msub>\u0000 </mfenced>\u0000 </mrow></math> is the probability of the outcome <span></span><math>\u0000 <mrow>\u0000 ","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1636","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524768","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Altmetric.com or PlumX: Does it matter?","authors":"Behrooz Rasuli, Majid Nabavi","doi":"10.1002/leap.1631","DOIUrl":"https://doi.org/10.1002/leap.1631","url":null,"abstract":"<p>Altmetric.com and PlumX are two prominent tools for collecting alternative metrics data. This study has two main objectives: first, to evaluate how the choice between Altmetric.com and PlumX affects the results of alternative metrics analysis, and second, to investigate the social impact of ‘hot papers’ through the alternative metrics data provided by these platforms. We employed a descriptive and exploratory approach, gathering common alternative metrics from 4236 hot papers using both Altmetric.com and PlumX. The data collected included various alternative metrics such as policy mentions, Mendeley readers, Wikipedia mentions, blog mentions, Facebook mentions, and news mentions, in addition to citation counts from Scopus. We conducted descriptive statistics and inferential analyses to examine the relationships between citations and alternative metrics, as well as to compare the data obtained from both platforms. Our findings indicate that PlumX has broader coverage of hot papers compared to Altmetric.com. While the mean and individual values of alternative metrics differ between the two platforms, the median and geometric mean are similar. Both Altmetric.com and PlumX demonstrate that publications with higher citation counts tend to receive more online attention. Notably, all alternative metrics for <i>Immunology</i> and <i>Chemistry</i> showed statistically significant differences between the two platforms, whereas in <i>Mathematics</i>, alternative metrics (with the exception of Mendeley readers) did not exhibit significant differences. The findings suggest that researchers should be aware of potential variations in data captured by different alternative metrics platforms. Additionally, interpreting alternative metrics data requires caution, considering the research fields and the specific platform used.</p>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1631","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Effects of the coronavirus 2019 pandemic on medical publishing: The sacrifice of quality for quantity?","authors":"Aliza Becker","doi":"10.1002/leap.1625","DOIUrl":"https://doi.org/10.1002/leap.1625","url":null,"abstract":"<p>Facing the coronavirus disease 2019 (COVID-19) pandemic, medical publishers rose to the occasion, moving to make their full portfolio of COVID-19–related research available to read for free and expediting peer review and production processes. With such a rapid transition from paper submission to publication, however, concerns also arose regarding whether the quality of the research publication process was being affected. This article seeks to document the transformation of medical publishers' practices in response to the COVID-19 pandemic and briefly discuss where they may go from here. For this goal, a literature search was performed in PubMed at several points to identify papers that reported early trends in how medical publishers handled COVID-19 research.</p>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1625","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
David Nicholas, Marzena Swigon, David Clark, Abdullah Abrizah, Jorge Revez, Eti Herman, Blanca Rodríguez Bravo, Jie Xu, Anthony Watkinson
{"title":"The impact of generative AI on the scholarly communications of early career researchers: An international, multi-disciplinary study","authors":"David Nicholas, Marzena Swigon, David Clark, Abdullah Abrizah, Jorge Revez, Eti Herman, Blanca Rodríguez Bravo, Jie Xu, Anthony Watkinson","doi":"10.1002/leap.1628","DOIUrl":"https://doi.org/10.1002/leap.1628","url":null,"abstract":"<p>The Harbingers study of early career researchers (ECRs), their work life and scholarly communications, began by studying generational—Millennial—change (c.2016), then moved to pandemic change (c.2020) and is now investigating another potential agent of change: artificial intelligence (2024–). We report here on a substantial scoping pilot study that looks at the impact of AI on the scholarly communications of international ECRs and, extends this to the arts and humanities. It aims to fill the knowledge gap concerning ECRs whose millennial mindset may render them especially open to change and, as the research workhorses they are, very much in the frontline. The data was collected via in-depth interviews in China, Malaysia, Poland, Portugal, Spain and (selectively) the United Kingdom/United States. The data show ECRs to be thinking, probing and, in some cases, experimenting with AI. There was a general acceptance that AI will be responsible for the growth of low-quality scientific papers, which could lead to a decline in the quality of research. Scholarly integrity and ethics were a big concern with issues of authenticity, plagiarism, copyright and poor citation practices raised. The most widespread belief was AI would prove to be a transformative force and would exacerbate existing scholarly disparities and inequalities.</p>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1628","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Field-specific gold open access dynamics in the Chinese mainland: Overviews, disparities, and strategic insights","authors":"Xinyi Chen, Zhiqiang Liu","doi":"10.1002/leap.1630","DOIUrl":"https://doi.org/10.1002/leap.1630","url":null,"abstract":"<p>Gold Open Access (OA) journals are crucial for scholarly communication, highlighting the need for a thorough evaluation of their academic influence on different research fields. This study leverages the InCites platform to examine article-level characteristics relating to 22 Essential Science Indicators (ESI) research fields, with a focus on the dynamics of gold OA articles, including gold OA uptake in the Chinese mainland and gold OA adoption in the domestic English-language academic journal publishing of the Chinese mainland. The findings reveal that disparities in gold OA adoption across 22 ESI fields are more pronounced in the Chinese mainland compared with the world scenario. In the Chinese mainland, there is a significant polarization in gold OA publishing volumes across different ESI fields, particularly in Chemistry, Clinical Medicine, and Engineering. This study builds on the understanding of OA citation advantage (OACA) by incorporating gold OA publishing volume into a two-dimensional framework, resulting in the development of a “distance” metric. It further categorizes gold OA citation effects into four quadrants: positive citation effects (quadrants A and B) and negative citation effects (quadrants C and D), based on category normalized citation impact (CNCI) and journal normalized citation impact (JNCI) indicators from the InCites database. The findings underscore the importance of developing tailored strategies to address field-specific challenges and promote gold OA dynamics in the Chinese mainland; while prioritizing high-quality gold OA journals is essential for fostering gold OA development in the rest of the world.</p>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1630","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
David Nicholas, Abdullah Abrizah, Jorge Revez, Blanca Rodríguez Bravo, Marzena Swigon, David Clark, Jie Xu, Anthony Watkinson, Eti Herman
{"title":"Are early career researchers feeling the consequences of the COVID pandemic?","authors":"David Nicholas, Abdullah Abrizah, Jorge Revez, Blanca Rodríguez Bravo, Marzena Swigon, David Clark, Jie Xu, Anthony Watkinson, Eti Herman","doi":"10.1002/leap.1629","DOIUrl":"https://doi.org/10.1002/leap.1629","url":null,"abstract":"<p>During the COVID pandemic, some commentators thought that early career researchers (ECRs) would become a ‘lost generation’. Yet the Harbingers (H-2) longitudinal study, which followed ECRs for 2 years during the pandemic found that ECRs took things in their stride. More than 2 years on, we returned, as part of the AI stage of the Harbingers study (H-3), to see what has transpired and interviewed nearly 70 ECRs from six countries as part of an exploratory study. We found that: (1) only one in six ECRS thought they were suffering from the residual impacts of the pandemic, with increased workload creating the greatest stress; (2) working remotely, digitally and more flexibly was thought by many to be the biggest dividend delivered, making the job more desirable and competitive; (3) an apparent growth in jobs and promotions (tenures) confirmed the bounce-back. Given the modest size of the study, analysis by country has to be undertaken with caution, but China, where political and economic factors have stifled any bounce-back, stands out for special attention in a future full-blown study. Finally, the study yielded informed and considered views that the long-term impact of the pandemic appears to be beneficial rather than disadvantageous.</p>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1629","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524526","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Min Dong, Wenjing Wang, Xuemei Liu, Fang Lei, Yunmei Luo
{"title":"Status of peer review guidelines in international surgical journals: A cross-sectional survey","authors":"Min Dong, Wenjing Wang, Xuemei Liu, Fang Lei, Yunmei Luo","doi":"10.1002/leap.1624","DOIUrl":"https://doi.org/10.1002/leap.1624","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Aim</h3>\u0000 \u0000 <p>To gain insight into the current status of peer review guidelines in international surgical journals and to offer guidance for the development of peer review guidelines for surgical journals.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We selected the top 100 journals in the category of ‘Surgery’ according to the Journal Citation Report 2021. We conducted a search of the websites of these journals, and Web of Science, PubMed, other databases, in order to gather the peer review guidelines published by these top 100 journals up until June 30, 2022. Additionally, we analysed the contents of these peer review guidelines.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Only 52% (52/100) of journals provided guidelines for reviewers. Sixteen peer review guidelines which were published by these 52 surgical journals were included in this study. The contents of these peer review guidelines were classified into 33 items. The most common item was research methodology, which was mentioned by 13 journals (25%, 13/52). Other important items include statistical methodology, mentioned by 11 journals (21.2%), the rationality of figures, tables, and data, mentioned by 11 journals (21.2%), innovation of research, mentioned by nine journals (17.3%), and language expression, readability of papers, ethical review, references, and so forth, mentioned by eight journals (15.4%). Two journals described items for quality assessment of peer review. Forty-three journals offered a checklist to guide reviewers on how to write a review report. Some surgical journals developed peer review guidelines for reviewers with different academic levels, such as professional reviewers and patient/public reviewers. Additionally, some surgical journals provided specific items for different types of papers, such as original articles, reviews, surgical videos, surgical database research, surgery-related outcome measurements, and case reports in their peer review guidelines.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Key contents of peer review guidelines for the reviewers of surgical journals not only include items relating to reviewing research methodology, statistical methods, figures, tables and data, research innovation, ethical review, but also cover items concerning reviewing surgical videos, surgical database research, surgery-related outcome measurements, instructions on how to write a review report, and guidelines on how to assess quality of peer review.</p>\u0000 </section>\u0000 </div>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1624","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Transformative agreements, publication venues and Open Access policies at the University of Milan","authors":"Laura Berni, Francesco Zucchini","doi":"10.1002/leap.1627","DOIUrl":"https://doi.org/10.1002/leap.1627","url":null,"abstract":"<p>Starting from July 2020 at the University of Milan, one of the largest and most important Italian universities, the first transformative agreements with some major international scientific publishers have come into effect. These agreements allow corresponding authors to publish in open access without directly bearing the publication costs. From the perspective of corresponding authors, these agreements could increase the dissemination of their scientific output and, thereby, the impact on the scientific community. However, transformative agreements are part of a rapidly changing publishing market that already includes open access articles in both so-called ‘Diamond’ and ‘Gold’ journals. The aim of our study is to understand to what extent the positioning of journals in impact rankings, the disciplinary field of the article, together with the career stage of the corresponding author, influence the choice to publish in a journal covered by transformative agreements rather than in other open access or hybrid journals. The results of our investigation draw attention to the importance of rules in Italy governing scientific careers in different disciplinary fields and potential unforeseen effects of policies favouring open access.</p>","PeriodicalId":51636,"journal":{"name":"Learned Publishing","volume":"37 4","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/leap.1627","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524869","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}