CoChat

Red flags for paper mills need to go beyond the level of individual articles: a case study of Hindawi special issues

Open access

The DOI matches its Crossref record.

Abstract

Background: Organisations known as paper mills charge authors to place fraudulent papers in the academic literature. Publishers have been slow to tackle the problem, but are now starting to devise methods for identifying paper mill products. However, little attention has been paid to the topic of complicit editors, who can take over special issues of journals and then publish many fraudulent articles. To date, activities of such editors have been documented by a handful of sleuths on social media and on the website PubPeer. This paper reports a descriptive study that documents more systematically the presence of “red flags” indicative of paper mill activity in special issues from the Wiley-Hindawi Open Access publishing partnership. Methods: A spreadsheet was created from the Hindawi website with records for all published articles during 2022. Initial analysis focused on initial Editor Response Time (RT) in ten journals that had been identified by sleuths as having high rates of problematic papers. In a second step, the whole dataset was scrutinized to identify editors who had handled at least 10 articles. These were divided into those who had had at least one PubPeer comment flagging dubious content or citations, and those with no PubPeer comments.Results: A cutoff of 22 days was identified as corresponding to the 2nd percentile of Editor RT for regular articles not in special issues. Plots show that in the 10 selected journals, some special issues have 50% or more of Editor RTs below this cutoff, raising questions as to whether an appropriate peer review process had taken place. Editors with articles flagged on PubPeer processed significantly more articles and had significantly shorter Editor RT than other editors. They often acted as editors in multiple issues. Other potential red flags, i.e., proportions of papers accepted without revision, or lack of diversity of author emails did not distinguish the two subgroups of editors. Discussion: Relatively simple indicators, of number of articles processed and speed of editorial processing, can be used as red flags that suggest an editor should be investigated to ensure they are not part of a paper mill. Conclusion: To tackle paper mills, it is crucial to move from a focus on individual articles to identify and root out individuals who have used positions as guest editors of special issues to flood the literature with fraudulent material. It is hoped that this analysis makes a start in suggesting approaches that can be used to this end.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.31234/osf.io/6mbgv

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Still being checked.

Research works printed but not linked: 18

  1. Abalkina, A., & Bishop, D. (2022). Paper mills: A novel form of publishing malpractice affecting psychology. PsyArXiv . https://doi.org/10.31234/osf.io/2yf8z · DOI
  2. Allen, M., Poggiali, D., Whitaker, K., Marshall, T. R., Van Langen, J., & Kievit, R. A. (2021). Raincloud plots: A multi-platform tool for robust data visualization [version 2; peer review: 2 approved]. Wellcome Open Research, 15191.2. https://doi.org/10.12688/wellcomeopenres.15191.2 · DOI · Paper page
  3. Byrne, J. A., & Christopher, J. (2020). Digital magic, or the dark arts of the 21st century—How can journals and peer reviewers detect manuscripts and publications from paper mills? FEBS Letters , 594 (4), 583–589. https://doi.org/10.1002/1873-3468.13747 · DOI · Paper page
  4. Byrne, J. A., & Labbé, C. (2017). Striking similarities between publications from China describing single gene knockdown experiments in human cancer cell lines. Scientometrics , 110(3), 1471–1493. https://doi.org/10.1007/s11192-016-2209-6 · DOI · Paper page
  5. Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology, 72(12), 1461–1476. https://doi.org/10.1002/asi.24495 · DOI · Paper page
  6. Committee on Publication Ethics (COPE) Council. (2019, November 2). Retraction guidelines . COPE: Committee on Publication Ethics. https://publicationethics.org/retraction-guidelines . DOI https://doi.org/10.24318/cope.2019.1.4 · DOI · Paper page
  7. COPE & STM. (2022). Paper mills: Research report from COPE & STM. Committee on Publication Ethics and STM. https://doi.org/10.24318/jtbG8IHL · DOI
  8. Else, H. (2021). Scammers impersonate guest editors to get sham papers published. Nature , 599 (7885), 361–361. https://doi.org/10.1038/d41586-021-03035-y · DOI
  9. Flaherty, C. (2022, June 13). The Peer-Review Crisis. Inside Higher Ed. https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars
  10. Fox, C. W. (2017). Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution. Scientometrics , 113 (1), 465–477. https://doi.org/10.1007/s11192-017-2489-5 · DOI
  11. Kincaid, E. (2022, September 28). Hindawi and Wiley to retract over 500 papers linked to peer review rings. Retraction Watch . https://retractionwatch.com/2022/09/28/exclusive-hindawi-and-wiley-to-retract-over-500-papers-linked-to-peer-review-rings/
  12. Kincaid, E. (2023, April 5). Hindawi and Wiley to retract 1,200 more papers for compromised peer review. Retraction Watch . https://retractionwatch.com/2023/04/05/wiley-and-hindawi-to-retract-1200-more-papers-for-compromised-peer-review/
  13. Ortega, J. L. (2022). Classification and analysis of PubPeer comments: How a web journal club is used. Journal of the Association for Information Science and Technology , 73 (5), 655–670. https://doi.org/10.1002/asi.24568 · DOI · Paper page
  14. Seifert, R. (2021). How Naunyn-Schmiedeberg's Archives of Pharmacology deals with fraudulent papers from paper mills. Naunyn-Schmiedeberg's Archives of Pharmacology , 394(3), 431–436. https://doi.org/10.1007/s00210-021-02056-8 · DOI · Paper page
  15. Smut Clyde, S., & Tiger BB8 (2019, June 4). Springer secretly ashamed, Elsevier lets it all hang out. For Better Science . https://forbetterscience.com/2019/06/04/springer-secretly-ashamed-elsevier-lets-it-all-hang-out/
  16. Teixeira da Silva, J. A. (2021a). Paper mill-derived cancer research: The improbability of prostate cancer in women, and ovarian and breast cancer in men. Nowotwory. Journal of Oncology , 71 (4), Article 4. https://doi.org/10.5603/NJO.a2021.0039 · doi:10.5603/njo.a2021.0039
  17. Teixeira da Silva, J. A. (2021b). Paper mills and on-demand publishing: Risks to the integrity of journal indexing and metrics. Medical Journal Armed Forces India , 77 (1), 119–120. https://doi.org/10.1016/j.mjafi.2020.08.003 · DOI
  18. Teixeira da Silva, J. A. (2022). When academic papers' stated emails do not match authors' affiliations: A new budding crisis in paper mill-ridden academic publishing? Epistēmēs Metron Logos , 8 , Article 8. https://doi.org/10.12681/eml.31441 · doi:10.12681/eml.31441

Cited by

See all 16

Metadata

Authors
Bishop, D. V. M.
Affiliations
University of Oxford (GB)
Corresponding author
Bishop, Dorothy Vera Margaret
Published
2023-02-06
OpenAlex
W4319444588
Cited by
16
References
0
Citations by year
2026: 1 · 2025: 2 · 2024: 8 · 2023: 5
Type
posted content
Language
en
Open access
Locations
  • PsyArXiv (OSF Preprints) (submittedVersion, public-domain)
  • PsyArXiv (OSF Preprints) (submittedVersion)
Topics
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
Keywords
paper mills, research misconduct, special issue, red flags, peer review, PubPeer, Hindawi, open access publishing, guest editors

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Red flags for paper mills need to go beyond the level of individual articles: a case study of Hindawi special issues. https://app.cochat.ai/papers/10.31234/osf.io/6mbgv
The paper
Bishop, D. V. M. (2023). Red flags for paper mills need to go beyond the level of individual articles: a case study of Hindawi special issues. https://doi.org/10.31234/osf.io/6mbgv

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.