Retracted papers originating from paper mills: a cross-sectional analysis of references and citations
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
OBJECTIVES: The aims of this study are (1) to analyze the references cited by retracted papers originated from paper mills; (2) to analyze the citations received by retracted papers originated from paper mills; and (3) to analyze the potential relationships existing between paper mill papers and their references and their citations. STUDY DESIGN AND SETTING: This study was a cross-sectional study. All original papers retracted in 2022 identified as having originated from paper mills and had been published at least 12 months before their retraction (hereinafter "source-retracted papers") were included. The Retraction Watch database was used to identify the source-retracted papers and Web of Science was used to identify the references contained within them and the citations received by them. We described the characteristics of the papers and journals. Additionally, 2 networks of source-retracted papers mutually interconnected via their citations and references were built: 1 with only retracted references and retracted citations and the other with all references and citations (retracted or unretracted). RESULTS: A total of 416 paper mill papers retracted in 2022 (sourced retracted papers) were identified, with a median of 1247 (interquartilic range, 907.8-1673.5) days between publication and retraction. Of all authors identified, 92.3% were affiliated with Chinese institutions. There were 14,411 references contained in the source-retracted papers and 8479 citations received by them; the median number of references and citations was 35 (29-40) and 16 (9-25), respectively. In total, 473 references and citations had also been retracted for being paper mill papers. Among the 416 sourced-retracted papers, 169 (41.9%) and 178 (42.8%) were referenced or were cited by at least another retracted paper, the majority of which also originated from paper mills. The first network analysis, which included source-retracted papers along with their retracted references and citations, found 3 clusters of 53, 48, and 44 retracted papers that were mutually interconnected. The second network analysis, with all references and citations (retracted or unretracted) identified a large cluster of 2530 interconnected papers. CONCLUSION: Retracted papers originating from paper mills frequently reference and are cited by papers that are later retracted for having originated from paper mills, displaying inter-relationships. Detecting these inter-relationships can serve as an indicator for identifying potentially fraudulent publications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The fight against fake-paper factories that churn out sham science
Else, H., & Van Noorden, R. (2021) · Nature · doi:10.1038/d41586-021-00733-5 · Cited by 251
Byrne, J. A., & Christopher, J. (2020) · FEBS Letters · doi:10.1002/1873-3468.13747 · Cited by 133
How big is science’s fake-paper problem?
Van Noorden, R. (2023) · Nature · doi:10.1038/d41586-023-03464-x · Cited by 114
The landscape of biomedical research
González-Márquez, R., Schmidt, L., Schmidt, B. M., Berens, P., & Kobak, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100968 · Cited by 89
Threats to scholarly research integrity arising from paper mills: a rapid scoping review
Pérez-Neri, I., Pineda, C., & Sandoval, H. (2022) · Clinical Rheumatology · doi:10.1007/s10067-022-06198-9 · Cited by 70
How Naunyn-Schmiedeberg’s Archives of Pharmacology deals with fraudulent papers from paper mills
Seifert, R. (2021) · Naunyn-Schmiedeberg's Archives of Pharmacology · doi:10.1007/s00210-021-02056-8 · Cited by 62
Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production
Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M. (2023) · doi:10.1101/2023.05.06.23289563 · Cited by 51
Paper-mill detector put to the test in push to stamp out fake science
Else, H. (2022) · Nature · doi:10.1038/d41586-022-04245-8 · Cited by 27
Sabel, B. A., & Seifert, R. (2021) · Naunyn-Schmiedeberg's Archives of Pharmacology · doi:10.1007/s00210-021-02158-3 · Cited by 27
Research works printed but not linked: 5
- Candal-Pedreira, 2022, Retracted papers originating from paper mills: cross sectional study, BMJ, 379
- Abalkina, 2021, Publication and collaboration anomalies in academic papers originating from a paper mill: evidence from Russia, arXiv
- Zhu, 2024, Citations of microRNA biomarker articles that were retracted: a systematic review, JAMA Netw Open, 7, 10.1001/jamanetworkopen.2024.3173 · DOI
- Cukier, 2020, Checklists to detect potential predatory biomedical journals: a systematic review, BMC Med, 18, 104, 10.1186/s12916-020-01566-1 · DOI · Paper page
- Heck, 2021, Fake data, paper mills, and their authors: the International Journal of Cancer reacts to this threat to scientific integrity, Int J Cancer, 149, 492, 10.1002/ijc.33604 · DOI · Paper page
Cited by
Paper mill challenges: past, present, and future
Parker, L., Boughton, S., Bero, L., & Byrne, J. A. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111549 · Cited by 81
Meho, L. I. (2025) · Scientometrics · doi:10.1007/s11192-025-05480-2 · Cited by 27
“Measures the proportion of an institution’s research output that has been retracted, serving as a critical marker of research integrity (Candal-Pedreira et al., 2024; Fang et al., 2012; Feng et al., 2024; Gedik et al., 2024; Ioannidis et al., 2025).”
Citation Contamination by Paper Mill Articles in Systematic Reviews of the Life Sciences
Tang, G., & Cai, H. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.15160 · Cited by 25
Berrío, J. P., & Kalliokoski, O. (2025) · FEBS Letters · doi:10.1002/1873-3468.70077 · Cited by 7
Metadata
- Authors
- Candal-Pedreira, C., Guerra-Tort, C., Ruano-Ravina, A., Freijedo-Farinas, F., Rey-Brandariz, J., Ross, J. S., & Pérez-Ríos, M.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2024-08
- Volume
- 172
- Pages
- 111397
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W4399107834
- Cited by
- 33
- References
- 9
- FWCI
- 13.62
- Type
- journal article
- Language
- en
- Status
- Retracted
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc/4.0/ (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Retracted papers originating from paper mills: a cross-sectional analysis of references and citations. https://app.cochat.ai/papers/10.1016/j.jclinepi.2024.111397
- The paper
- Candal-Pedreira, C., Guerra-Tort, C., Ruano-Ravina, A., Freijedo-Farinas, F., Rey-Brandariz, J., Ross, J. S., & Pérez-Ríos, M. (2024). Retracted papers originating from paper mills: a cross-sectional analysis of references and citations. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2024.111397
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.