Red flags for paper mills need to go beyond the level of individual articles: a case study of Hindawi special issues
Open accessThe DOI matches its Crossref record.
Abstract
Background: Organisations known as paper mills charge authors to place fraudulent papers in the academic literature. Publishers have been slow to tackle the problem, but are now starting to devise methods for identifying paper mill products. However, little attention has been paid to the topic of complicit editors, who can take over special issues of journals and then publish many fraudulent articles. To date, activities of such editors have been documented by a handful of sleuths on social media and on the website PubPeer. This paper reports a descriptive study that documents more systematically the presence of “red flags” indicative of paper mill activity in special issues from the Wiley-Hindawi Open Access publishing partnership. Methods: A spreadsheet was created from the Hindawi website with records for all published articles during 2022. Initial analysis focused on initial Editor Response Time (RT) in ten journals that had been identified by sleuths as having high rates of problematic papers. In a second step, the whole dataset was scrutinized to identify editors who had handled at least 10 articles. These were divided into those who had had at least one PubPeer comment flagging dubious content or citations, and those with no PubPeer comments.Results: A cutoff of 22 days was identified as corresponding to the 2nd percentile of Editor RT for regular articles not in special issues. Plots show that in the 10 selected journals, some special issues have 50% or more of Editor RTs below this cutoff, raising questions as to whether an appropriate peer review process had taken place. Editors with articles flagged on PubPeer processed significantly more articles and had significantly shorter Editor RT than other editors. They often acted as editors in multiple issues. Other potential red flags, i.e., proportions of papers accepted without revision, or lack of diversity of author emails did not distinguish the two subgroups of editors. Discussion: Relatively simple indicators, of number of articles processed and speed of editorial processing, can be used as red flags that suggest an editor should be investigated to ensure they are not part of a paper mill. Conclusion: To tackle paper mills, it is crucial to move from a focus on individual articles to identify and root out individuals who have used positions as guest editors of special issues to flood the literature with fraudulent material. It is hoped that this analysis makes a start in suggesting approaches that can be used to this end.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Short handling time is regarded by some as a hallmark of paper mill activity (68, 69).”
From The entities enabling scientific fraud at scale are large, resilient, and growing rapidly (2025) · cites this paper as [68]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Still being checked.
Research works printed but not linked: 18
- Abalkina, A., & Bishop, D. (2022). Paper mills: A novel form of publishing malpractice affecting psychology. PsyArXiv . https://doi.org/10.31234/osf.io/2yf8z · DOI
- Allen, M., Poggiali, D., Whitaker, K., Marshall, T. R., Van Langen, J., & Kievit, R. A. (2021). Raincloud plots: A multi-platform tool for robust data visualization [version 2; peer review: 2 approved]. Wellcome Open Research, 15191.2. https://doi.org/10.12688/wellcomeopenres.15191.2 · DOI · Paper page
- Byrne, J. A., & Christopher, J. (2020). Digital magic, or the dark arts of the 21st century—How can journals and peer reviewers detect manuscripts and publications from paper mills? FEBS Letters , 594 (4), 583–589. https://doi.org/10.1002/1873-3468.13747 · DOI · Paper page
- Byrne, J. A., & Labbé, C. (2017). Striking similarities between publications from China describing single gene knockdown experiments in human cancer cell lines. Scientometrics , 110(3), 1471–1493. https://doi.org/10.1007/s11192-016-2209-6 · DOI · Paper page
- Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology, 72(12), 1461–1476. https://doi.org/10.1002/asi.24495 · DOI · Paper page
- Committee on Publication Ethics (COPE) Council. (2019, November 2). Retraction guidelines . COPE: Committee on Publication Ethics. https://publicationethics.org/retraction-guidelines . DOI https://doi.org/10.24318/cope.2019.1.4 · DOI · Paper page
- COPE & STM. (2022). Paper mills: Research report from COPE & STM. Committee on Publication Ethics and STM. https://doi.org/10.24318/jtbG8IHL · DOI
- Else, H. (2021). Scammers impersonate guest editors to get sham papers published. Nature , 599 (7885), 361–361. https://doi.org/10.1038/d41586-021-03035-y · DOI
- Flaherty, C. (2022, June 13). The Peer-Review Crisis. Inside Higher Ed. https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars
- Fox, C. W. (2017). Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution. Scientometrics , 113 (1), 465–477. https://doi.org/10.1007/s11192-017-2489-5 · DOI
- Kincaid, E. (2022, September 28). Hindawi and Wiley to retract over 500 papers linked to peer review rings. Retraction Watch . https://retractionwatch.com/2022/09/28/exclusive-hindawi-and-wiley-to-retract-over-500-papers-linked-to-peer-review-rings/
- Kincaid, E. (2023, April 5). Hindawi and Wiley to retract 1,200 more papers for compromised peer review. Retraction Watch . https://retractionwatch.com/2023/04/05/wiley-and-hindawi-to-retract-1200-more-papers-for-compromised-peer-review/
- Ortega, J. L. (2022). Classification and analysis of PubPeer comments: How a web journal club is used. Journal of the Association for Information Science and Technology , 73 (5), 655–670. https://doi.org/10.1002/asi.24568 · DOI · Paper page
- Seifert, R. (2021). How Naunyn-Schmiedeberg's Archives of Pharmacology deals with fraudulent papers from paper mills. Naunyn-Schmiedeberg's Archives of Pharmacology , 394(3), 431–436. https://doi.org/10.1007/s00210-021-02056-8 · DOI · Paper page
- Smut Clyde, S., & Tiger BB8 (2019, June 4). Springer secretly ashamed, Elsevier lets it all hang out. For Better Science . https://forbetterscience.com/2019/06/04/springer-secretly-ashamed-elsevier-lets-it-all-hang-out/
- Teixeira da Silva, J. A. (2021a). Paper mill-derived cancer research: The improbability of prostate cancer in women, and ovarian and breast cancer in men. Nowotwory. Journal of Oncology , 71 (4), Article 4. https://doi.org/10.5603/NJO.a2021.0039 · doi:10.5603/njo.a2021.0039
- Teixeira da Silva, J. A. (2021b). Paper mills and on-demand publishing: Risks to the integrity of journal indexing and metrics. Medical Journal Armed Forces India , 77 (1), 119–120. https://doi.org/10.1016/j.mjafi.2020.08.003 · DOI
- Teixeira da Silva, J. A. (2022). When academic papers' stated emails do not match authors' affiliations: A new budding crisis in paper mill-ridden academic publishing? Epistēmēs Metron Logos , 8 , Article 8. https://doi.org/10.12681/eml.31441 · doi:10.12681/eml.31441
Cited by
The strain on scientific publishing
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2024) · Quantitative Science Studies · doi:10.1162/qss_a_00327 · Cited by 345
The strain on scientific publishing
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2023) · arXiv · doi:10.48550/arxiv.2309.15884 · Cited by 345
The entities enabling scientific fraud at scale are large, resilient, and growing rapidly
Richardson, R. A. K., Hong, S. S., Byrne, J. A., Stoeger, T., & Amaral, L. A. N. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2420092122 · Cited by 129
Paper mill challenges: past, present, and future
Parker, L., Boughton, S., Bero, L., & Byrne, J. A. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111549 · Cited by 81
Metadata
- Authors
- Bishop, D. V. M.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Bishop, Dorothy Vera Margaret
- Publisher
- Center for Open Science
- Published
- 2023-02-06
- OpenAlex
- W4319444588
- Cited by
- 16
- References
- 0
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 8 · 2023: 5
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- PsyArXiv (OSF Preprints) (submittedVersion, public-domain)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- paper mills, research misconduct, special issue, red flags, peer review, PubPeer, Hindawi, open access publishing, guest editors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Red flags for paper mills need to go beyond the level of individual articles: a case study of Hindawi special issues. https://app.cochat.ai/papers/10.31234/osf.io/6mbgv
- The paper
- Bishop, D. V. M. (2023). Red flags for paper mills need to go beyond the level of individual articles: a case study of Hindawi special issues. https://doi.org/10.31234/osf.io/6mbgv
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.