Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.

Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production

Open access
Sabel, Bernhard A., Knaack, Emely, Gigerenzer, Gerd, Bilc, Mirela · 2023 · Naunyn-Schmiedeberg s Archives of Pharmacology

Abstract

Background: Integrity of academic publishing is increasingly undermined by fake science publications massively produced by commercial “editing services” (so-called “paper mills”). They use AI-supported, automated production techniques at scale and sell fake publications to students, scientists, and physicians under pressure to advance their careers. Because the scale of fake publications in biomedicine is unknown, we developed a simple method to red-flag them and estimate their number. Methods: To identify indicators able to red-flag fake publications (RFPs), we sent questionnaires to authors. Based on author responses, a classification rule was applied initially using the two-indicators “non-institutional email AND no international authors” (“email+NIA”) to sub-samples of 15,120 PubMed®-listed publications regarding publication date, journal, impact factor, country and RFP citations. Using the indicator “hospital affiliation” (“email+hospital”), this classification (tallying) rule was validated by comparing 400 known fakes with 400 matched presumed non-fakes. Results: Two initial indicators (“email+NIA”) revealed a rapid rise of RFP from 2010 to 2020. Countries with the highest RFP proportion were Russia, Turkey, China, Egypt, India and China (39%-55%). When using the “email+hospital” tallying-rule, sensitivity of RFP identification was 86%, the false alarm rate 44%, and the estimated RFP rate in 2020 was 11.0%. Adding a RFP-citation indicator (“email+hospital+RFP-citations”) increased the sensitivity to 90% and reduced the false alarm rate to 37%. Given 1.3 million biomedical Scimago-listed publications, the estimated annual RFP number in 2020 is about 150,000. Conclusions: Potential fake publications can be red-flagged using simple-to-use, validated classification rules to earmark them for subsequent scrutiny. RFP rates are increasing, suggesting higher actual fake rates than previously reported. The large scale and proliferation of fake publications in biomedicine can damage trust in science, endanger public health, and impact economic spending and security. Easy-to-apply fake detection methods, as proposed here, or more complex automated methods can enable the retraction of fake publications at scale and help prevent further damage to the permanent scientific record.

Overview

Integrity signals

Registry

Crossref · 10.1101/2023.05.06.23289563

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Figures

Fig. 1 Screening plan for journals and publications. Studies 1–6 identified potential indicators of fake publications on the basis of hypotheses 1 to 3 (see text), using the indicators “private email” and “no international author”, Studies 7 used the indicator combination “email” and “hospital” as an alternative classification rule. To estimate the incidence of potential fake publications in Studies 8 and 9, we used a more complex multifactorial tallying rule. Note that some publications were used in several screening studies. Total number of unique publications analyzed: n = 17,120
Fig. 1 Screening plan for journals and publications. Studies 1–6 identified potential indicators of fake publications on the basis of hypotheses 1 to 3 (see text), using the indicators “private email” and “no international author”, Studies 7 used the indicator combination “email” and “hospital” as an alternative classification rule. To estimate the incidence of potential fake publications in Studies 8 and 9, we used a more complex multifactorial tallying rule. Note that some publications were used in several screening studies. Total number of unique publications analyzed: n = 17,120CC BY · PubMed Central
Fig. 2 2D landscape map of biomedical topics of 21 million biomedical publications. A Color patches represent different disciplines (topics) in basic science to the left of the white line (e.g., immunology, physiology, and chemistry) and in clinical science to the right (e.g., pharmacology, neurology, and radiology) (for details, see González-Márquez et al. 2023). The position of actual paper mill publications obtained by one of us (B.S.) (98 red triangles in A) shows that the large majority are from clinical and not basic science disciplines. The paper mill publications in the 12 o’clock cluster are from cancer research and the cluster in the center from microbiology/virology. B Distribution of publications with hospital affiliation only (black) or hospital affiliation and private email address (red). The distribution shows that private email addresses are widespread across fields, forming clusters in some topics but not in others. The private/institutional email address ratio is 894,124/5,800,617 = 15.4%
Fig. 2 2D landscape map of biomedical topics of 21 million biomedical publications. A Color patches represent different disciplines (topics) in basic science to the left of the white line (e.g., immunology, physiology, and chemistry) and in clinical science to the right (e.g., pharmacology, neurology, and radiology) (for details, see González-Márquez et al. 2023). The position of actual paper mill publications obtained by one of us (B.S.) (98 red triangles in A) shows that the large majority are from clinical and not basic science disciplines. The paper mill publications in the 12 o’clock cluster are from cancer research and the cluster in the center from microbiology/virology. B Distribution of publications with hospital affiliation only (black) or hospital affiliation and private email address (red). The distribution shows that private email addresses are widespread across fields, forming clusters in some topics but not in others. The private/institutional email address ratio is 894,124/5,800,617 = 15.4%CC BY · PubMed Central
Fig. 3 The rise of red-flagged fake publications (RFPs). Number of publications and percentage of red-flagged fake publications in medicine (top) and neuroscience (bottom) (Study 3). The red line (y1-axis) shows the percentage of red-flagged publications per year; the grey bars (y2-axis) show the estimates of non-fakes, and the red bars show those of potentially fake publications, extrapolated to overall articles published in journals of the Neuroscience Peer Review Consortium and in medicine (selected from scimagojr.com). Note the relatively slow increase in potential fakes in medicine and the rapid rise in neuroscience
Fig. 3 The rise of red-flagged fake publications (RFPs). Number of publications and percentage of red-flagged fake publications in medicine (top) and neuroscience (bottom) (Study 3). The red line (y1-axis) shows the percentage of red-flagged publications per year; the grey bars (y2-axis) show the estimates of non-fakes, and the red bars show those of potentially fake publications, extrapolated to overall articles published in journals of the Neuroscience Peer Review Consortium and in medicine (selected from scimagojr.com). Note the relatively slow increase in potential fakes in medicine and the rapid rise in neuroscienceCC BY · PubMed Central

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

3 retracted

Show all 51 references

Research works printed but not linked: 2

  1. Abalkina A (2021) Publication and collaboration anomalies in academic papers originating from a paper mill: evidence from Russia arXiv preprint arXiv.2112.13322 · arXiv
  2. 2013, An empirical analysis on number and monetary value of ghostwritten papers in, China Current Science, 105, 1230

Cited by

See all 78

Metadata

Authors
Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M.
Affiliations
  • Center for Behavioral Brain Sciences (DE)
  • Otto-von-Guericke-Universität Magdeburg (DE)
  • Max Planck Institute for Human Development (DE)
Corresponding author
Sabel, Bernhard A.
Venue
Naunyn-Schmiedeberg s Archives of Pharmacology
Publisher
openRxiv
Published
2023-05-08
Volume
399
Issue
2
Pages
2943-2955
ISSN
0028-1298 (ISSN-L)
OpenAlex
W4414490687
Cited by
51
References
51
Citations by year
2026: 12 · 2025: 15 · 2024: 17 · 2023: 7
FWCI
27.94
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Naunyn-Schmiedeberg s Archives of Pharmacology (publishedVersion, cc-by)
  • medRxiv (acceptedVersion, cc-by-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
Funders
  • Eberhard Karls Universität Tübingen
  • Otto von Guericke University Magdeburg
Topics
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
Keywords
paper mills, research integrity, bibliometric analysis, Bayes' theorem, publication fraud, biomedical literature, retraction, AI-generated content
MeSH terms
Humans, Periodicals as Topic, Publishing, Surveys and Questionnaires, Bibliometrics, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production. https://app.cochat.ai/papers/10.1101/2023.05.06.23289563
The paper
Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M. (2023). Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production. Naunyn-Schmiedeberg s Archives of Pharmacology. https://doi.org/10.1101/2023.05.06.23289563

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.