Collusion between journals, editors, authors leads to alarming growth of research fraud
The number of retracted articles and papers commented for misconduct on PubPeer has been doubling every 3.3 years and 3.6 years, respectively, while papers produced by paper mills have been doubling every 1.5 years
It is beyond doubt that peer-reviewing system, the cornerstone of scientific publishing, is broken. Hundreds and thousands of peer-reviewed papers with multiple manipulated images, same images reused in multiple papers dealing with completely different materials, flooding of papers produced by ‘paper mills’ — fraudulent organisations that produce and sell mass-produced low quality and fabricated research articles — and papers generated using Large Language Models (LLMs) without proper disclosure at the time of submission are already well known.
Now, using data from PLOS ONE and Hindawi journals, a paper published in the journal Proceedings of the National Academy of Sciences (PNAS) has produced damning evidence and digital footprint of a widespread network of authors, editors, and reviewers colluding to publish papers bypassing the traditional peer-reviewing standards set by journal publishers. Not only that, the authors of the PNAS study were able to find evidence of organisations that ensure the publication of fraudulent papers going one step further. The paper mills ensure that questionable papers escape the existing science integrity and quality control measures, such as journal deindexing. Finally, the study found an alarming trend — the number of fraudulent papers that get published is “growing at a rate far exceeding the growth of legitimate publications”.
Scientific literature swamped by fraudulent papers
Retractions have been increasing sharply in recent years. Notably, retractions by journals in the past couple of years are done in batches of more than 10 papers at a time with identical reasons cited in the retraction notice of each paper. Even as the number of retracted articles and papers commented for misconduct on PubPeer has been doubling every 3.3 years and every 3.6 years, respectively, the total number of publications has been doubling every 15 years. In comparison, papers produced by paper mills have been doubling every 1.5 years. “Suspected paper mill products now outnumber annually retracted articles and are projected to soon outnumber the number of PubPeer-commented articles,” the PNAS authors write.
Only about 8,600 of the about 30,000 suspected paper mill papers have been retracted (28.7%). “Extrapolating from current trends, we estimate that only around 25% [or one in four] of suspected paper mill papers will ever be retracted, and that only around 10% of suspected paper mill products will ever reside in a deindexed journal,” they write.
Spotlight on India for the wrong reasons
Academic Research Development Association (ARDA), a Chennai-based entity is in the business of conducting “Conferences and Meetings”, “Journal Publications” and “Thesis/Article Writing” the website says. Since 2018, ARDA has grown from only 14 journals to 86 journals in March 2024, with 17 journals suspected to be “hijacked” journals — where a journal was once legitimate but a paper mill has gained complete editorial control over the journal and its indexed content! The PNAS paper says that the “evolution of ARDA’s portfolio of journals” is ominous — new titles appear the moment old ones are deindexed, negating the very purpose of deindexing. The rate at which new titles are added in direct response to deindexing is apparently very high and far exceeds the baseline rate. “To our knowledge, this is the first reported case of an entity engaging in fraudulent publishing that itself engages in journal hopping,” they write referring to how a group of journals deindexed by SCOPUS in 2020 or 2021 were subsequently replaced by new journals. Many papers published in ARDA journals have no connection with the scope of the journal. They cite two examples — an article about roasting hazelnuts in a journal about HIV/AIDS care, and an article about malware detection in a journal about special education. The two examples cited are not isolated cases but rather a norm with ARDA journals. The reason: between 34% and 98.7% of the “papers” published in these journals were “outside of the journal’s stated scope”
Over 10% of papers published had authors from multiple countries, “supporting the hypothesis that paper mills will sell authorship slots on individual manuscripts”. In a damning charge, the authors write that “paper mills, predatory journals, and brokers likely operate under a number of author procurement models”, wherein local scholars are targeted with no international authorship and another model where authorship is sold to scholars across the world. Of the 13,288 papers published in five journals — The Journal of Pharmaceutical Negative Results, PalArch’s Journal of Archaeology of Egypt/Egyptology, Res Militaris, Russian Law Journal and, HIV Nursing — for which the researchers established the nationality, 26.4% were from India, 19.3% were from Iraq, and 12.2% were from Indonesia.
“We download the archives of five journals on ARDA’s menu of offerings and show that these journals have collectively published more than 20,000 articles from around the world, most of which are well outside of the stated scope of the journal,” Dr. Reese Richardson from Northwestern University, Evanston, Illinois, U.S. and the first author of the PNAS paper writes in his blog.
How the study was conducted
The PNAS authors used the data from PLOS ONE and Hindawi journals that disclose the names of the handling editor of every published paper. They downloaded metadata of articles published in PLOS ONE by November 8, 2023, and of articles published in Hindawi journals by April 2, 2024. They also collected reports of image duplication made on PubPeer, where science integrity researchers point out potential misconduct in published papers. In May 2023, the complete archive of five journals listed on the Academic Research and Development Association (ARDA) website was downloaded. Finally, the PNAS authors compiled a corpus of suspected paper mill products comprising 32,786 unique papers. The presence of tortured phrases in papers was used as a criterion to identify paper mill papers.
While the general perception is that scientific fraud is rare and arises from isolated bad actors, evidence suggests otherwise — the possibility of fraud is not only more pervasive but also bad actors target certain journals to ensure publication of fraudulent science at scale. To investigate this, the PNAS authors matched the names of editors responsible for accepted manuscripts who are more likely to edit problematic articles, which goes beyond chance.
Role of rogue editors and authors
PLOS ONE, which discloses the names of the handling editors, has published 2,76,956 articles since 2006, of which 702 have been retracted and 2,241 have received comments on PubPeer. From the 18,329 handling editors who have accepted articles for publication in PLOS ONE, the PNAS authors determined the number of papers that each handing editor had accepted for publication, and the number of accepted papers that were eventually retracted each year. Here’s what they found — 22 handling editors had accepted articles that were eventually retracted “significantly more frequently” than by chance alone. And when the PNAS authors took into account the papers that were commented upon for misconduct on PubPeer, the list of handling editors increased from 22 to 33.
Based on the data of the handling editors who were red flagged for accepting problematic papers, the PNAS authors next investigated whether papers by certain authors were more frequently assigned to one of the red-flagged editors more than expected by chance. And here’s what they found — 21 authors had papers more frequently assigned to editors flagged for retractions, and papers of 18 authors were assigned to editors flagged for comments on PubPeer.
In the case of PLOS ONE, anomalous peer-review, wherein papers are accepted for publication in less than 30 days after submission — considered as a hallmark of paper mill activity — was found to be a less robust marker to identify potential collusion compared with retractions and comments about misconduct made on PubPeer.
The PNAS paper found a “densely connected group of individuals serving as editors between 2020 and 2023” sent most of their submissions to one another rather than to other editors. What is indeed striking is that more than half of the papers accepted by these closely-connected editors have been retracted. Remarkably, the retraction notices on each of these retracted papers are nearly identical — “one of a series of submissions for which we have concerns about authorship, competing interests, and peer review”.
How Hindawi journals fared
Besides PLOS ONE, the PNAS authors had also carried out similar pattern investigation for 10 Hindawi journals too; PLOS ONE and Hindawi journals disclose the names of the handling editors, the reason why the PNAS authors chose to study them for collusion by bad actors. This is what they found — 53 editors accepted papers that were eventually retracted, 52 editors accepted papers that were commented for misconduct on PubPeer and 96 editors accepted papers for publication in less than 30 days after submission.
Applying the same methodology to all IEEE conference proceedings published since 2003, the PNAS authors flagged 84 conferences for hosting eventually retracted articles “anomalously frequently”, 158 conferences for hosting articles that were commented upon PubPeer anomalously frequently, and 183 conferences for hosting articles with tortured phrases “anomalously frequently”.
Duplicated images to ascertain paper mills’ role
To investigate if there is collusion between journals and paper mills, and if that leaves a digital trace in the scientific literature, the PNAS authors looked for image duplication, which is considered a hallmark of fraudulent science. The hypothesis was that paper mills produce and publish articles in large batches and therefore may use a fixed bank of images. And such papers would be heavily concentrated within specific publishers and within specific years. The results proved that their hypothesis was indeed correct — the publication dates and the journals publishing the papers containing duplicated images were extremely concentrated in time and by publishers.
“Our working hypothesis and these anomalous patterns are consistent with a modus operandi where paper mills cooperate with brokers — or act, themselves, as brokers — who control at least some of the decisions at target journals and can guarantee the simultaneous publication of batches of fraudulent articles in a single journal,” they write.
Frauds within disciplinary subfields
The PNAS authors restricted their analysis to papers on closely-related fields in RNA biology, which have become widely popular in recent times. Within the field of RNA, the PNAS authors further narrowed down to six subfields — CRISPR-Cas9, transfer RNAS (tRNAs) and development, tRNAs and cancer, circular RNAs, micro-RNAs (miRNAs) and development, miRNAs and cancer, and long noncoding RNAs (lncRNAs).
The publication rate of errata for all subfields was found to vary between 1.5% and 2.5%, while the rate of retractions varies from about 0.1% for CRISPR-Cas9 to a peak of about 4% in the case of miRNAs and lncRNAs. “Our findings suggest that some fields are indeed more vulnerable to widespread contamination of their literature than others. We surmise that paper mills have a preference for fields in which plausible manuscripts can be generated rapidly by mixing and matching novel elements into formulaic templates,” Dr. Richardson writes in his blog.
