Red flags for paper mills need to go beyond the level of individual articles: a case study of Hindawi special issues
Open accessThe DOI matches its Crossref record; the references it cites are not checked yet.
Abstract
Background: Organisations known as paper mills charge authors to place fraudulent papers in the academic literature. Publishers have been slow to tackle the problem, but are now starting to devise methods for identifying paper mill products. However, little attention has been paid to the topic of complicit editors, who can take over special issues of journals and then publish many fraudulent articles. To date, activities of such editors have been documented by a handful of sleuths on social media and on the website PubPeer. This paper reports a descriptive study that documents more systematically the presence of “red flags” indicative of paper mill activity in special issues from the Wiley-Hindawi Open Access publishing partnership. Methods: A spreadsheet was created from the Hindawi website with records for all published articles during 2022. Initial analysis focused on initial Editor Response Time (RT) in ten journals that had been identified by sleuths as having high rates of problematic papers. In a second step, the whole dataset was scrutinized to identify editors who had handled at least 10 articles. These were divided into those who had had at least one PubPeer comment flagging dubious content or citations, and those with no PubPeer comments.Results: A cutoff of 22 days was identified as corresponding to the 2nd percentile of Editor RT for regular articles not in special issues. Plots show that in the 10 selected journals, some special issues have 50% or more of Editor RTs below this cutoff, raising questions as to whether an appropriate peer review process had taken place. Editors with articles flagged on PubPeer processed significantly more articles and had significantly shorter Editor RT than other editors. They often acted as editors in multiple issues. Other potential red flags, i.e., proportions of papers accepted without revision, or lack of diversity of author emails did not distinguish the two subgroups of editors. Discussion: Relatively simple indicators, of number of articles processed and speed of editorial processing, can be used as red flags that suggest an editor should be investigated to ensure they are not part of a paper mill. Conclusion: To tackle paper mills, it is crucial to move from a focus on individual articles to identify and root out individuals who have used positions as guest editors of special issues to flood the literature with fraudulent material. It is hoped that this analysis makes a start in suggesting approaches that can be used to this end.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Short handling time is regarded by some as a hallmark of paper mill activity (68, 69).”
From The entities enabling scientific fraud at scale are large, resilient, and growing rapidly (2025) · cites this paper as [68]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 6
- Flaherty, C. (2022, June 13). The Peer-Review Crisis. Inside Higher Ed. https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars
- Kincaid, E. (2022, September 28). Hindawi and Wiley to retract over 500 papers linked to peer review rings. Retraction Watch . https://retractionwatch.com/2022/09/28/exclusive-hindawi-and-wiley-to-retract-over-500-papers-linked-to-peer-review-rings/
- Kincaid, E. (2023, April 5). Hindawi and Wiley to retract 1,200 more papers for compromised peer review. Retraction Watch . https://retractionwatch.com/2023/04/05/wiley-and-hindawi-to-retract-1200-more-papers-for-compromised-peer-review/
- Smut Clyde, S., & Tiger BB8 (2019, June 4). Springer secretly ashamed, Elsevier lets it all hang out. For Better Science . https://forbetterscience.com/2019/06/04/springer-secretly-ashamed-elsevier-lets-it-all-hang-out/
- Teixeira da Silva, J. A. (2021a). Paper mill-derived cancer research: The improbability of prostate cancer in women, and ovarian and breast cancer in men. Nowotwory. Journal of Oncology , 71 (4), Article 4. https://doi.org/10.5603/NJO.a2021.0039 · doi:10.5603/njo.a2021.0039
- Teixeira da Silva, J. A. (2022). When academic papers' stated emails do not match authors' affiliations: A new budding crisis in paper mill-ridden academic publishing? Epistēmēs Metron Logos , 8 , Article 8. https://doi.org/10.12681/eml.31441 · doi:10.12681/eml.31441
Cited by
The strain on scientific publishing
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2024) · Quantitative Science Studies · doi:10.1162/qss_a_00327 · Cited by 345
The strain on scientific publishing
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2023) · arXiv · doi:10.48550/arxiv.2309.15884 · Cited by 345
The entities enabling scientific fraud at scale are large, resilient, and growing rapidly
Richardson, R. A. K., Hong, S. S., Byrne, J. A., Stoeger, T., & Amaral, L. A. N. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2420092122 · Cited by 129
Paper mill challenges: past, present, and future
Parker, L., Boughton, S., Bero, L., & Byrne, J. A. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111549 · Cited by 81
La ruta de oro de la publicación científica: del negocio de las revistas a las revistas negocio
López‐Cózar, E. D., & Martín‐Martín, A. (2024) · Revista Mediterránea de Comunicación · doi:10.14198/medcom.26763 · Cited by 34
Paper mills: a novel form of publishing malpractice affecting psychology
Bishop, D., & Abalkina, A. (2023) · Meta-Psychology · doi:10.15626/mp.2022.3422 · Cited by 28
Self-citationMetadata analysis of retracted fake papers in Naunyn-Schmiedeberg’s Archives of Pharmacology
Wittau, J., & Seifert, R. (2023) · Naunyn-Schmiedeberg s Archives of Pharmacology · doi:10.1007/s00210-023-02850-6 · Cited by 17
López‐Cózar, E. D., & Martín‐Martín, A. (2024) · Anales de química · doi:10.62534/rseq.aq.1946 · Cited by 12
Repiso, R., & Delgado-Vázquez, Á. M. (2024) · Revista Mediterránea de Comunicación · doi:10.14198/medcom.25211 · Cited by 11
Smagulov, K., & da Silva, J. A. T. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-025-09624-0 · Cited by 9
The Issue with Special Issues: when Guest Editors Publish in Support of SelfPreprint
Crosetto, P., Barreiro, P. G., & Hanson, M. A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.07563 · Cited by 9
Paper mills, fraudulent authors, and editorial responses
Smart, P. (2023) · Science Editing · doi:10.6087/kcse.314 · Cited by 6
Retractions as a Bitter Pill Corrective Measure to Eliminate Flawed Science
Joshi, P. B., & Minirani, S. (2024) · Springer eBooks · doi:10.1007/978-981-97-4060-4_18 · Cited by 3
Harness editors’ networks of communication to fight publication fraud
Hu, G. (2023) · Accountability in Research · doi:10.1080/08989621.2023.2199931 · Cited by 3
Tracking the retracted paper mill articles: a bibliometric study
Cheng, M., Xinhua, Y., & Allen, R. M. (2026) · Scientometrics · doi:10.1007/s11192-026-05751-6 · Cited by 2
Commentary: Fifty years exploring pharmacology with Sam Enna
Williams, M. J. (2024) · Biochemical Pharmacology · doi:10.1016/j.bcp.2024.116067 · Cited by 2
Analyzing the use of email addresses in scholarly publications
Luwel, M., & van Eck, N. J. (2023) · International Conference on Science, Technology and Innovation Indicators (STI) · doi:10.55835/644287b5b8e4087ea9272b0f · Cited by 2
Eysmondt, C. V. (2024) · API Magazin · doi:10.15460/apimagazin.2024.5.1.187 · Cited by 1
Paper mills, fraudulent authors, and editorial responses
Smart, P. (2024) · Science Editor and Publisher · doi:10.24069/sep-23-19 · Cited by 1
Spelling errors in health and medical abstracts: An observational analysis from 2008 to 2024Preprint
Barnett, A. G. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/v8n9q_v2 · Cited by 1
A do-or-die moment for the scientific enterprise
Richardson, R. (2025) · doi:10.59350/ahdx7-bsg75 · Cited by 1
The Dark Sides of Modern Science: Publishing and Dissemination (Part II)Preprint
Sochi, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5507278
Pohlmann, T. (2025) · O-bib · doi:10.17170/kobra-2025031410966
Repiso, R., & Delgado-Vázquez, Á. M. (2023) · Dialnet (Universidad de la Rioja) · doi:10.5281/zenodo.7787812
Metadata
- Authors
- Bishop, D. V. M.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Bishop, Dorothy Vera Margaret
- Publisher
- Center for Open Science
- Published
- 2023-02-06
- OpenAlex
- W4319444588
- Cited by
- 16
- References
- 0
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 8 · 2023: 5
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- PsyArXiv (OSF Preprints) (submittedVersion, public-domain)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- paper mills, research misconduct, special issue, red flags, peer review, PubPeer, Hindawi, open access publishing, guest editors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.