What is a paper mill?

Abstract

In this editorial, I will attempt to capture some thoughts for reflection and discussion on the science publication process, as it currently presents to researchers. Publication—communication of findings—is both a natural sequel to and an integral part of research, and critical to researchers' career development. Unfortunately, while there are many positives, there are also some significant emerging issues needing to be resolved in today's science publication sector. From a philosophical perspective, the aim of the publishing process should be to distill and communicate the key new information generated by a research project. Publication has an archival function (creating a record of what was done), a networking function (allowing researchers in the same field to learn from and build on each other's findings), and a knowledge building function (contributing to the sum of human knowledge).

A bogey word often raised in recent years is ‘paper mill’, defined as the fraudulent creation of superficially normal manuscripts for sale to authors who are prepared to use such tactics to enhance their CVs and advance their careers. A recent analysis in Nature (Van Noorden, 2023) indicated that 3% of all articles published in medicine and biology in the last two decades are likely to be paper mill products. A second current concern is ‘predatory publishing’ defined as the collection of publication fees without the normal quality and integrity controls associated with scientific publication.

The scientific publication sector is large, and researchers often don't appreciate just how large. Among the well-known Scientific publishers, Springer have 3000+, Taylor and Frances 2700+, Elsevier 2600+ and Wiley 1600+ journal titles, according to publishers' own websites. MDPI, regarded by some but not by the writer as a predatory publisher has 473 journal titles. In their 2023 Annual Report, MDPI indicate 655 000 papers submitted and 285 244 published in that year, with 1.4 million peer review reports received. Internet sources indicate over 5 million scientific papers per year published currently with 744 000 papers from China and 624 000 from the United States in journals listed by Scopus in 2020. Considering that a typical article processing charge for open access publication is in the range of $US 2000–3500 and journals using a subscription model presumably have a similar revenue, it is immediately clear that the total financial turnover associated with the global scientific publishing sector is similar to the GDP of a medium sized country, such as Australia.

The scientific publication sector has also grown and evolved dramatically over the last 50 years. A Clarivate Web of Science search by the writer for papers with the topic ‘grassland’ returned 423 articles in the period 1971–1975, 11 459 articles in the period 1996–2000, and 37 939 articles in the last 5 years. Corresponding numbers for papers from China were 0, 192 and 14 056. Over the same period, the typical journal ownership structure has also changed. In the 1970s, it was common for a journal to be owned by a government department or a scientific society. Now, most titles are owned, or at least managed by commercial publishing houses. For example, the New Zealand Journal of Agricultural Research was formerly published by the New Zealand government Department of Scientific and Industrial Research and is now published by Taylor and Francis; the Journal of the British Grassland Society which was published by the Society from its offices changed its name to Grass and Forage Science, and publication was subsequently transferred to Wiley. If we simplistically consider the sector as comprising publishers, authors and stakeholders such as universities, funding agencies and government science administrators we can begin to logically deduce some of the positive outcomes and negative pressures that will now be emerging.

Publishers will be strongly motivated to increase revenue which may come either from increasing charges, or consolidating and rationalizing operations and reducing services where possible to reduce costs. They will also be concerned to achieve prestige within the sector and approval of authors who publish with them, to retain market share. This will usually involve careful adherence to principles of integrity and considerations like sound peer review and reducing publication time. Authors will be most interested in readership and status of their published work with their peers and colleagues, and in considerations like rapid publication. The level of understanding of their work displayed by peer reviewers and whether or not feedback is constructive will also be important. Stakeholders are often interested to measure researcher outputs for employment or promotion decisions. This means that publisher metrics like impact factor can be expected to impact disproportionately on author behaviour.

From the writer's perspective as an editor of Grassland Research, the evolution described above in the publishing sector has brought both positives and negatives. On the positive side, the old days of waiting months or even more than a year for a first decision on a submitted manuscript are gone. Journals now compete with one another to achieve decision times measured in days.

A negative outcome which will not be explored in detail here because of the sensitivity, is the current high cost of publication. Factors such as the value to authors of achieving publication in a prestigious journal and the prestige value to institutions of holding a complete portfolio of journals in their library have provided an environment where commercial publishing houses can increase subscription and article processing costs. Suffice to say, escalating subscription and publishing costs have become a significant issue for authors and institutions around the world in recent years. It appears to the writer that in some cases publication gross revenues may exceed 400% of publishers' direct costs, and while it is certainly legitimate to exploit commercial opportunity, it is equally not good for the long-term wellbeing of the sector for one group to be taking overly high profit from another group within the sector. The writer is aware of various universities reducing journal title holdings in recent years in response to escalation of publisher charges, and this will inevitably impact negatively on future student access to information, and ultimately on research quality. Also, cost reduction through avenues like transferring typesetting services to countries with comparatively low labour costs can mean a reduced level of scientific expertise in the proofing stages, with a potential increase in the rate of grammatical and other errors in published articles.

A second area where evolution of the science publication sector over recent decades has led to obvious issues is in the manuscript review process. Journals including Grassland Research currently struggle with a problem of researcher reluctance to accept invitations to review manuscripts. In the 1970s, it was typically the case that a researcher in a country such as New Zealand, once appointed, could expect to retain their job until retirement with substantial personal choice about the direction of their research program and comparatively relaxed reporting requirements. Invitations to review were seldom refused and were undertaken as a part of one's job with the time paid by the employer. In 2020s, researchers typically bid competitively to funding agencies. These bids include the cost of the researchers' time and employment may be terminated where there is no funding. In this scenario, a researcher accepting a review invitation is effectively working for free in their own time, quite often outside of normal work hours.

For commercial publishing houses to continue to expect reviews to be performed for free as they used to be, is unreasonable. Ironically, a reviewer submitting a peer review report in their own time in the late evening after their family has gone to bed, or after an early morning session before breakfast, will typically receive a banal, computer-generated email message a few seconds later assuring them that their contribution is important and appreciated by the journal! It really is not surprising that a growing number of researchers are routinely declining to accept review invitations. In response to growing reviewer reluctance, many publishers now encourage their handling editors to simultaneously invite multiple reviewers when a manuscript is first received, so that the required number of reviews will be met quickly. That stance is disrespectful of reviewer time, as it creates a risk that unneeded reviews will be procured. From a reverse perspective, publishers regard it as misconduct when an author submits a manuscript simultaneously to multiple journals with a view to publishing with the journal offering the first or easiest acceptance. The writer senses a double standard here. A step change in publisher practice is needed on the issue of fair compensation for reviewers for their input to the publication process.

At the other end of the reviewer spectrum, cursory reviews that fail to detect a serious issue with a manuscript because the reviewer didn't have the expertise to identify the problem or merely because the reviewer assumed a manuscript section would be correct and didn't check it, or reviews that display misunderstanding of the researchers' work are now becoming an issue for editorial offices to detect and deal with. When a flawed manuscript is accepted, a journal's standing with other authors is likely to be diminished; similarly no editor wants to disappoint an author by returning a review with unreasonable comments or revision requests. Care and professionalism need to be exercised when a researcher does accept a review invitation. Another facet of the problem is that many publishing house staff are trained in business or communication rather than research science and struggle to see the true extent of the issues with reviewer reluctance and poor quality of some reviews, faced by their editorial offices.

A third point of concern is the impact on author behaviour of the pressure to produce publications. In extreme cases, authors resort to concrete misconduct such as alteration or fabrication of data. There is no need to dwell on that here; everyone engaged in science knows that such behaviour is unacceptable. However, when it comes to author behaviour, the various scenarios cannot be neatly classified into right and wrong. Between black and white is every shade of grey. A detailed analysis of problematic author behaviour in publication and the pressures that lead to it was presented at the November 2024 Grassland Research International Forum on Grassland Research by Professor Johannes Knops from Xi'an Jiaotong-Liverpool University. Issues traversed in that analysis include, among others, confusion between correlation and causation, reporting only significant results out of multiple comparisons, pseudo-replication where repeat samples are taken within the same patch or plot and non-reproducibility. Non-reproducibility can occur where sampling, though replicated, is for some reason not truly representative of the effect the researcher intended to study. For example, when using RNA sequencing to study the composition of soil microbial communities, a sampled effect may be temporally variable, meaning that resampling would produce different results. An issue periodically noted by the writer is mathematically descriptive inclusion of multiple multivariate analyses of the same data (e.g., a principal component analysis (PCA), a cluster analysis and a structural equation model) to give the impression of sophistication, but without attempt at interpretation of the biological meaning of the data or the interrelationship between the analyses. When a PCA and cluster analysis are performed on the same data set it is likely that one or more principal components will differentiate between different groups in the cluster analysis. Many more examples could be cited. Authors, supervisors of graduate students, science administrators, and publishers all have a role to play in reducing the incidence of problems of this type with manuscripts.

For now, if we define a paper mill product as any manuscript where the focus is shifted away from reporting novel research findings for archival reasons or to increase the sum of human knowledge, towards generation of a paper purely to create a measurable research output, then the proportion of papers affected in our discipline is much higher than the 3% estimated by Van Noorden (2023). We all need to be working to identify occurrences, understand the reasons for them, and doing our part to correct the problem.