The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index; 12 of the 37 sources its claims cite were checked for support.
Fake publications in biomedical science: red-flagging method indicates mass production
Open accessAbstract
Integrity of academic publishing is increasingly undermined by fake publications massively produced by commercial “editing services” (so-called “paper mills”). These services use AI-supported production techniques at scale and sell fake publications to students, scientists, and physicians under pressure to advance their careers. Because the scale of fake publications in biomedicine is unknown, we developed an easy-to-apply rule to red-flag potentially fake publications and estimate their number. After analyzing questionnaires sent to authors of published papers, we developed simple classification rules and tested them in a 9-step bibliometric analysis in a sample of 17,120 publications listed in PubMed®. We first validated various simple rules and finally applied a multifactorial tallying rule comparing 400 known fakes with 400 random (presumed) non-fakes. This rule was then applied to 1,000 random publications each from 2020 and 2023. The multifactorial tallying rule was the best red-flagging method, with a 94% sensitivity and only a 11.5% false-alarm rate. The rate of red-flagged articles increased during the last decade, reaching an estimated 14.9% in 2020 and 16.3% in 2023. Countries with the highest proportion of read-flagged publications were China, India, Iran, Russia, and Turkey, with China and India the largest absolute contributors globally. Applying Bayes’ rule resulted in an estimate of 5.8% actual fakes in the biomedical literature. Given 1.86 million Scimago-listed biomedical publications in 2023, we estimate the actual number of true fakes at 107.800 articles per year, growing steadily. Scientific publications in biomedicine can be red-flagged as potentially fake using fast-and-frugal classification rules to earmark them for subsequent scrutiny. When applying Bayes´rule, the annual true scale of fake publishing in biomedicine is about 19 times that of the 5.671 biomedicine retractions in 2023. This scale of fraudulent publishing is concerning as it can damage trust in science, endanger public health, and impact economic spending. But fake detection tools can enable retractions of fake publications at scale and help prevent further damage to the permanent scientific record.
Overview
CoChat Summary
TL;DR
The paper develops and validates a fast, multifactorial red-flagging rule for detecting potentially fake biomedical publications, reporting 94% sensitivity and an 11.5% false-alarm rate against 400 known fakes and 400 presumed non-fakes. Applying it to 1,000 publications each from 2020 and 2023 yields red-flag rates of 14.9% and 16.3%, and Bayes’ rule estimates 5.8% true fakes, or about 107,800 articles per year in 2023.
Key findings
- The multifactorial tallying rule was the best red-flagging method, with 94% sensitivity and an 11.5% false-alarm rate; the simpler email-hospital rule had 85% sensitivity and false-alarm rates of 1% (2000) or 8% (2010).
- Red-flagged publication rates increased over the last decade, reaching 14.9% in 2020 and 16.3% in 2023. Countries with the highest within-country proportions were Russia (66.7%), India (54.2%), China (38.7%), Iran (29.6%), and Turkey (20.8%); China (42.3%) and India (33.0%) were the largest between-country contributors.
- Using Bayes’ rule, the estimated rate of true fakes was 5.8%, with a positive predictive value of a red flag of 33.5%. Given 1.86 million Scimago-listed biomedical publications in 2023, the estimated annual number of true fakes was 107,800, about 19 times the 5,671 biomedical retractions in 2023.
- A paper mill interview provided 98 publications, 96 of which were red-flagged by the indicators; paper mills advertised fees of $1,000–$25,000 per publication, and the estimated annual paper mill revenue in biomedicine was about $1 billion in 2023.
Methods
- The study used a 9-step bibliometric analysis in a sample of 17,120 publications listed in PubMed and Web of Science in neuroscience and medicine, beginning with questionnaires sent to corresponding authors of suspected and control articles.
- Validation compared 400 known fakes (100 paper-mill retractions, 100 tadpole-like image plagiarisms, 100 fake gene sequences, and 100 retractions from the Journal of Cellular Biochemistry) with 400 presumed non-fakes from 2000 and 2010. The multifactorial tallying rule red-flagged publications scoring >2 points, with points for private email, hospital affiliation, suspicious country, and low institutional output, and points subtracted for co-author number and high institutional output.
- The validated rule was applied to 1,000 publications from 2020 (10 journals) and 1,000 from 2023 (random keyword search selecting the first PubMed item); Bayes’ rule estimated the true fake rate. A 2D landscape map of 21 million biomedical abstracts using PubMedBERT was also used, along with analysis of >1,000 editing-service websites and a paper mill interview.
Limitations
- The paper states that journal and topic selection limits the study to biomedicine, and that the sample of 17,120 screened publications is several magnitudes smaller than the 1.86 million biomedical publications in 2023 alone.
- The paper states that the tallying method has an 11.5% false-alarm rate and 6% false negatives but does not allow an unambiguous judgment for a given paper; the probability of a true fake given a red flag is 33.5%, and the red-flagging method does not directly translate to other science disciplines.
- The paper states that the control publications were presumed non-fake and one cannot be certain each control was indeed not fake, though the probability was deemed low for 2000 and 2010; it also notes that the true fake estimate depends on the specific indicators, rules, and databases surveyed.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
11 Valid · 3 Not valid · 13 Not checked · 70 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“They were clustered in certain fields (cancer, virology) but not in others (for details regarding the different academic disciplines, see González-Márquez et al. 2024).”
§ Fake publications in biomedical science: red-flagging method indicates mass production > Results > Validation of fake indicators
What the sources say
- Partly supportedcites González-Márquez et al. 2024 sourceExperimental — accuracy not yet measured
In Results, ‘Retracted papers highlight suspicious literature,’ the paper reports that retracted papers cluster in specific areas, particularly cancer-related research, and identifies those areas as known paper-mill targets. It does not describe a set of 98 paper-mill publications or establish the claimed pattern for virology versus other fields. (LLM, conf=0.78)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Paper mills also feed on the rising worldwide administrative practice of evaluating researchers by the publish-or-perish criteria of counting papers and JIF ranking metrics as a surrogate for evaluating research quality and content (Van Dalen and Henkens 2012; Candal-Pedreira 2022).”
§ Fake publications in biomedical science: red-flagging method indicates mass production > Discussion
What the sources say
- Not checked — no full text of the sourcecites Van Dalen and Henkens 2012 source
- Partly supportedcites Candal-Pedreira 2022 sourceExperimental — accuracy not yet measured
The Introduction says paper mill papers artificially inflate researchers’ curricula, and the Discussion (Principal findings) reports publication incentives tied to money and career promotion in China. The paper does not establish a rising worldwide administrative practice of evaluating researchers by paper counts and journal impact factor (JIF) as substitutes for research quality; it uses impact factor to describe journal prestige, not researcher evaluation. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“After assessing the citations of 12 retracted clinical trial reports in 68 systematic reviews, meta-analyses, guidelines, and clinical trials, they concluded that 13 of the latter would likely have to change their conclusions without the data from the retracted publications.”
§ Fake publications in biomedical science: red-flagging method indicates mass production > Discussion
What the sources say
- Partly supportedcites Avenell et al. (2019) sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observed that publications fulfilling these three criteria shared easy-to-detect fake-indicators: The corresponding author had a private (non-institutional) email address, no international co-author, and/or a hospital affiliation.”
§ Fake publications in biomedical science: red-flagging method indicates mass production > Methods > Exploration
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When applying Bayes´rule, the annual true scale of fake publishing in biomedicine is about 19 times that of the 5.671 biomedicine retractions in 2023.”
§ Fake publications in biomedical science: red-flagging method indicates mass production > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
25 more claims
Create a free account to see all 97 claims and every source verdict.
- It may state two separate claims as one. 55
- Probably not a substantial claim. 25
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“this was topped only by open access (OA) Frontiers journals in Study 6 (112/300 = 40.3%).”
§ Fake publications in biomedical science: red-flagging method indicates mass production > Results > Exploration of fake indicators
Reported 40.3 · recomputed 37.333333
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Given the 61 million publications in all science disciplines and all years (https://www.scimagojr.com/worldreport.php), the percentage of roughly 50,000 known fakes (retractions) is 0.00082%.”
§ Fake publications in biomedical science: red-flagging method indicates mass production > Introduction
Reported 0.00082 · recomputed 0.081967
Cited works that were retracted
Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.
A note, not an issue: for example, a cited work that was retracted only after this paper was published, which its authors could not have known.
In the paper
“A questionnaire was sent to the authors and, for control, to 48 authors of unsuspicious papers with questions that authors of fake papers might be reluctant to answer (e.g., “Are you willing to provide original data if so asked?””
Cited work 10.1007/s11192-021-03895-1 · retraction date not known · this paper published 2025-09-24
A note, not an issue: for example, a cited work that was retracted only after this paper was published, which its authors could not have known.
In the paper
“Brown SJ, Bakker CJ, Theis-Mahon NR (2022) Retracted publications in pharmacy systematic reviews. J Med Libr Assoc 110:47–55”
Cited work 10.5195/jmla.2022.1280 · retraction date not known · this paper published 2025-09-24
A note, not an issue: for example, a cited work that was retracted only after this paper was published, which its authors could not have known.
In the paper
“Candal-Pedreira C (2022) Retracted papers originating from paper mills: cross sectional study. BMJ 379:e071517”
Cited work 10.1136/bmj-2022-071517 · retraction date not known · this paper published 2025-09-24
Citations
Cites
3 retracted
Retracted papers originating from paper mills: cross sectional study
Candal-Pedreira, C., Ross, J. S., Ruano-Ravina, A., Egilman, D. S., Fernández, E., & Pérez-Ríos, M. (2022) · BMJ · doi:10.1136/bmj-2022-071517 · Cited by 156
RetractedRetracted articles in the biomedical literature from Indian authors
Elango, B. (2021) · Scientometrics · doi:10.1007/s11192-021-03895-1 · Cited by 62 · cited 1×
“A questionnaire was sent to the authors and, for control, to 48 authors of unsuspicious papers with questions that authors of fake papers might be reluctant to answer (e.g., “Are you willing to provide original data if so asked?””
RetractedRetracted publications in pharmacy systematic reviews
Brown, S. J., Bakker, C., & Theis-Mahon, N. R. (2022) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2022.1280 · Cited by 41
RetractedMisconduct accounts for the majority of retracted scientific publications
Fang, F. C., Steen, R. G., & Casadevall, A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1212247109 · Cited by 1,232
Prevalence of Articles With Honorary Authors and Ghost Authors in Peer-Reviewed Medical Journals
Flanagin, A. (1998) · JAMA · doi:10.1001/jama.280.3.222 · Cited by 553 · cited 3×
“…in China (Hu and Wu 2013) and the 1% listed in Table 1, but similar to the 5–10% reported in a pharmacology (Seifert 2021) and a cancer journal (Heck et al. 2021), and below the 21–32% honorary, guest, and ghost authorships in biomedicine (Else 2021; Flanagin et al. 1998; Wislar et al. 2011).”
Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey
Wislar, J. S., Flanagin, A., Fontanarosa, P. B., & DeAngelis, C. D. (2011) · BMJ · doi:10.1136/bmj.d6128 · Cited by 478 · cited 5×
“The private/institutional email address ratio is 894,124/5,800,617 = 15.4% manuscript or publication alone.”
InfluentialIntended and unintended consequences of a publish‐or‐perish culture: A worldwide survey
van Dalen, H. P., & Henkens, K. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22636 · Cited by 355 · cited 3×
“We need fewer articles of better quality and should evaluate the content, quality, and relevance of research (Van Dalen and Henkens 2012).”
The Prevalence of Inappropriate Image Duplication in Biomedical Research Publications
Bik, E. M., Casadevall, A., & Fang, F. C. (2016) · mBio · doi:10.1128/mbio.00809-16 · Cited by 287 · cited 2×
“…researchers include ghost and “honorary” authorships (Flanagin et al., 1998; Wislar et al., 2011; Frederickson and Herzog, 2021), cherry-picking, abstract spin, plagiarism of images (Bik et al., 2016), and outright data fabrication (Bik, 2020; Byrne and Christopher, 2020; Park et al., 2022).”
Statistical Rituals: The Replication Delusion and How We Got There
Gigerenzer, G. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918771329 · Cited by 263 · cited 1×
“The “replication crisis” slows down the development of life-saving therapies with an estimated financial loss of $28 billion annually by the pharmaceutical industry (Gigerenzer, 2018).”
Self-citationThe fight against fake-paper factories that churn out sham science
Else, H., & Van Noorden, R. (2021) · Nature · doi:10.1038/d41586-021-00733-5 · Cited by 251 · cited 2×
“Trust in the integrity of academic publishing is a foundation of science, and lack of it damages its reputation (Behl 2021; Byrne 2019; Seifert 2021; Else and Van Noorden 2021; Else 2022; Byrne et al. 2022).”
Research works printed but not linked: 1
- Hu Z-w, Wu Y (2013) An empirical analysis on number and monetary value of ghostwritten papers in China. Curr Sci 105:1230–1234
Cited by
The entities enabling scientific fraud at scale are large, resilient, and growing rapidly
Richardson, R. A. K., Hong, S. S., Byrne, J. A., Stoeger, T., & Amaral, L. A. N. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2420092122 · Cited by 129
Kendall, G., & Teixeira da Silva, J. A. (2024) · Learned Publishing · doi:10.1002/leap.1578 · Cited by 100
The landscape of biomedical research
González-Márquez, R., Schmidt, L., Schmidt, B. M., Berens, P., & Kobak, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100968 · Cited by 89
“Using their database, we identified an additional 3,572 papers in our map that were not marked as retracted in PubMed but were in fact retracted (red dots in Figure 6).”
The misalignment of incentives in academic publishing and implications for journal reform
Trueblood, J. S., Allison, D. B., Field, S. M., Fishbach, A., Gaillard, S. D. M., Gigerenzer, G., Holmes, W. R., Lewandowsky, S., Matzke, D., Murphy, M. C., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401231121 · Cited by 56
Self-citationGenerative Artificial Intelligence in Anatomic Pathology
Brodsky, V., Ullah, E., Bychkov, A., Song, A. H., Walk, E. E., Louis, P., Rasool, G., Singh, R. S., Mahmood, F., Bui, M. M., et al. (2025) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2024-0215-ra · Cited by 48
Metadata
- Authors
- Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M.-I.
- Affiliations
- Center for Behavioral Brain Sciences (DE)
- Otto-von-Guericke-Universität Magdeburg (DE)
- Max Planck Institute for Human Development (DE)
- Corresponding author
- Sabel, Bernhard A.
- Venue
- Naunyn-Schmiedeberg's Archives of Pharmacology
- Published
- 2025-09-24
- Volume
- 399
- Issue
- 2
- Pages
- 2943-2955
- ISSN
- 0028-1298 (ISSN-L)
- 1432-1912
- PMC
- PMC12901079
- OpenAlex
- W4414490687
- Cited by
- 51
- References
- 47
- Citations by year
- 2026: 12 · 2025: 15
- FWCI
- 27.79
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Naunyn-Schmiedeberg s Archives of Pharmacology (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- Funders
- State of Sachsen-Anhalt Ministry of Science, Energy, Climate Protection and Environment (FKZ I 167)
- Otto-von-Guericke-Universität Magdeburg
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- paper mills, research integrity, bibliometric analysis, Bayes' theorem, publication fraud, biomedical literature, retraction, AI-generated content
- MeSH terms
- Humans, Periodicals as Topic, Publishing, Surveys and Questionnaires, Bibliometrics, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fake publications in biomedical science: red-flagging method indicates mass production. https://app.cochat.ai/papers/10.1007/s00210-025-04275-9
- The paper
- Sabel, B. A., Knaack, E., Gigerenzer, G., & Bilc, M.-I. (2026). Fake publications in biomedical science: red-flagging method indicates mass production. Naunyn-Schmiedeberg's Archives of Pharmacology. https://doi.org/10.1007/s00210-025-04275-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.