Exploratory analysis of text duplication in peer-review reveals peer-review fraud and paper mills
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Comments received from referees during peer-review were analysed to determine the rates of duplication and partial duplication. It is very unusual for 2 different referees to submit identical comments, so the rare cases where this happens are of interest. In some cases, it appears that paper-mills create fake referee accounts and use them to submit fake peer-review reports. These include comments that are copied and pasted across multiple reviews. Searching for duplication in referee comments is therefore an effective method to search for misconduct generally, since the forms of misconduct committed by paper-mills go beyond peer-review fraud. These search methods allow the automatic detection of misconduct candidates which may then be investigated carefully to confirm if misconduct has indeed taken place. There are innocent reasons why referees might share template reports, so these methods are not intended to automatically diagnose misconduct.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Prevalence of Inappropriate Image Duplication in Biomedical Research Publications
Bik, E. M., Casadevall, A., & Fang, F. C. (2016) · mBio · doi:10.1128/mbio.00809-16 · Cited by 287
Byrne, J. A., & Christopher, J. (2020) · FEBS Letters · doi:10.1002/1873-3468.13747 · Cited by 133 · cited 1×
“In recent years, fraudulent research being produced industrially by paper-mills has become a growing problem (Byrne & Christopher, 2020).”
BackgroundCan We Automate Scientific Reviewing?
Yuan, W., Liu, P., & Neubig, G. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12862 · Cited by 96
Retractions in medicine: the tip of the iceberg
Oransky, I., Fremes, S. E., Kurlansky, P., & Gaudino, M. (2021) · European Heart Journal · doi:10.1093/eurheartj/ehab398 · Cited by 30
Abuse of ORCID’s weaknesses by authors who use paper mills
Teixeira da Silva, J. A. (2021) · Scientometrics · doi:10.1007/s11192-021-03996-x · Cited by 27 · cited 1×
“However, there is little research on automated methods to detect or prevent this form of fraud, e.g. (Rivera & Teixeira da Silva, 2021).”
Method
Cited by
Sarstedt, M., Adler, S. J., Rau, L., & Schmitt, B. (2024) · Psychology & Marketing · doi:10.1002/mar.21982 · Cited by 149
“Publishers, journal editors, and reviewers need to be aware of this tug ‐ of ‐ war, which other fields are already witnessing in the context of paper mills (Candal ‐ Pedreira et al., 2022; Day, 2022; Pérez ‐ Neri et al., 2022).”
BackgroundAbalkina, A. (2023) · Learned Publishing · doi:10.1002/leap.1574 · Cited by 101
Retracted‘Stamp out paper mills’ — science sleuths on how to fight fake research
Abalkina, A., Aquarius, R., Bik, E., Bimler, D., Bishop, D., Byrne, J., Cabanac, G., Day, A., Labbé, C., & Wise, N. (2025) · Nature · doi:10.1038/d41586-025-00212-1 · Cited by 59
Self-citationDetection of fake papers in the era of artificial intelligence
Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023) · Diagnosis · doi:10.1515/dx-2023-0090 · Cited by 24
Metadata
- Authors
- Day, A.
- Venue
- Scientometrics
- Published
- 2022-09-08
- Volume
- 127
- Issue
- 10
- Pages
- 5965-5987
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- OpenAlex
- W4295977437
- Cited by
- 42
- References
- 5
- FWCI
- 10.52
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Exploratory analysis of text duplication in peer-review reveals peer-review fraud and paper mills. https://app.cochat.ai/papers/10.1007/s11192-022-04504-5
- The paper
- Day, A. (2022). Exploratory analysis of text duplication in peer-review reveals peer-review fraud and paper mills. Scientometrics. https://doi.org/10.1007/s11192-022-04504-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.