Explosion of formulaic research articles, including inappropriate study designs and false discoveries, based on the NHANES US national health database
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
With the growth of artificial intelligence (AI)-ready datasets such as the National Health and Nutrition Examination Survey (NHANES), new opportunities for data-driven research are being created, but also generating risks of data exploitation by paper mills. In this work, we focus on two areas of potential concern for AI-supported research efforts. First, we describe the production of large numbers of formulaic single-factor analyses, relating single predictors to specific health conditions, where multifactorial approaches would be more appropriate. Employing AI-supported single-factor approaches removes context from research, fails to capture interactions, avoids false discovery correction, and is an approach that can easily be adopted by paper mills. Second, we identify risks of selective data usage, such as analyzing limited date ranges or cohort subsets without clear justification, suggestive of data dredging, and post-hoc hypothesis formation. Using a systematic literature search for single-factor analyses, we identified 341 NHANES-derived research papers published over the past decade, each proposing an association between a predictor and a health condition from the wide range contained within NHANES. We found evidence that research failed to take account of multifactorial relationships, that manuscripts did not account for the risks of false discoveries, and that researchers selectively extracted data from NHANES rather than utilizing the full range of data available. Given the explosion of AI-assisted productivity in published manuscripts (the systematic search strategy used here identified an average of 4 papers per annum from 2014 to 2021, but 190 in 2024–9 October alone), we highlight a set of best practices to address these concerns, aimed at researchers, data controllers, publishers, and peer reviewers, to encourage improved statistical practices and mitigate the risks of paper mills using AI-assisted workflows to introduce low-quality manuscripts to the scientific literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A recent investigation revealed systematic fabrication of research manuscripts using NHANES data, resulting in hundreds of retractions [6].”
From Citation Hallucination Determines Success: An Empirical Comparison of Six Medical AI Research Systems (2026) · cites this paper as [6]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · cited 1×
“In addition, the ability to generate large numbers of machine-learning models allows for rapid post-hoc investigation of alternative hypotheses, should the main a priori hypothesis not be supported (a form of hypothesizing after the results are known, or HARKing) [10–12].”
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · cited 2×
“It is not possible to know which are examples of hypothesizing after the results are known (HARKing) [10,11], but the lack of disclosure and justification of choosing subsets is clearly a shortfall versus best practice.”
Atlas of plasma NMR biomarkers for health and disease in 118,461 individuals from the UK Biobank
Julkunen, H., Cichońska, A., Tiainen, M., Koskela, H., Nybo, K., Mäkelä, V., Nokso-Koivisto, J., Kristiansson, K., Perola, M., Salomaa, V., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-36231-7 · Cited by 536 · cited 2×
“This has previously been demonstrated in UK Biobank data, where latent systemic connectivity was found in common biomarkers across a wide range of health conditions [9].”
Ten things to know about ten cardiovascular disease risk factors
Bays, H. E., Taub, P. R., Epstein, E., Michos, E. D., Ferraro, R. A., Bailey, A. L., Kelli, H. M., Ferdinand, K. C., Echols, M. R., Weintraub, H., et al. (2021) · American Journal of Preventive Cardiology · doi:10.1016/j.ajpc.2021.100149 · Cited by 274
The fight against fake-paper factories that churn out sham science
Else, H., & Van Noorden, R. (2021) · Nature · doi:10.1038/d41586-021-00733-5 · Cited by 251 · cited 1×
“While this may be a product of different incentives in China leading to misconduct [18,19], it may also reflect the ability of small networks or groups of individuals to exploit AI-ready datasets for end-to-end generation of very large numbers of manuscripts.”
Rubin, M. (2017) · Review of General Psychology · doi:10.1037/gpr0000128 · Cited by 182 · cited 1×
“It is not possible to know which are examples of hypothesizing after the results are known (HARKing) [10,11], but the lack of disclosure and justification of choosing subsets is clearly a shortfall versus best practice.”
Byrne, J. A., & Christopher, J. (2020) · FEBS Letters · doi:10.1002/1873-3468.13747 · Cited by 133
Data-Dredging Procedures in Survey Analysis
Selvin, H., & Stuart, A. (1966) · The American Statistician · doi:10.1080/00031305.1966.10480401 · Cited by 132
Alberts, B., Hanson, B., & Kelner, K. L. (2008) · Science · doi:10.1126/science.1162115 · Cited by 119 · cited 1×
“…such as Committee on Publication Ethics (COPE) provide support and guidance to editors, publishers, and researchers [30], often peer reviewers will be the last, and therefore the most important, line of defence against poor practices [31], including those that may be adopted by paper mills.”
How to save costs by reducing unnecessary testing: Lean thinking in clinical practice
Vegting, I. L., van Beneden, M., Kramer, M. H. H., Thijs, A., Kostense, P. J., & Nanayakkara, P. W. B. (2011) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2011.07.003 · Cited by 110 · cited 1×
“Best practice is to conduct research in a comprehensive manner, or run the risk of research not being translational [23], with real costs to healthcare systems and patients [24,25].”
Turrini, S., Wong, B., Eldaief, M. C., Press, D. Z., Sinclair, D. A., Koch, G., Avenanti, A., & Santarnecchi, E. (2023) · Ageing Research Reviews · doi:10.1016/j.arr.2023.101939 · Cited by 99 · cited 1×
“…example, depression, cardiovascular disease, and cognitive function are all extensively described in the literature as multifactorial in nature [20–22], and yet were investigated in the papers included in this systematic review using single-factor approaches focusing on one predictor, albeit…”
(2022) · doi:10.24318/jtbg8ihl · Cited by 93 · cited 1×
“While there are challenges in common with those posed by manufactured manuscripts using falsified data [32,33], data dredging from open access datasets creates different issues (as the statistics and data may—in isolation—be sound), and will require different countervailing measures.”
nhanesA: achieving transparency and reproducibility in NHANES research
Ale, L., Gentleman, R., Sonmez, T. F., Sarkar, D., & Endres, C. (2024) · Database J. Biol. Databases Curation · doi:10.1093/database/baae028 · Cited by 57
Reclaiming mendelian randomization from the deluge of papers and misleading findings
Stender, S., Gellert‐Kristensen, H., & Smith, G. D. (2024) · Lipids in Health and Disease · doi:10.1186/s12944-024-02284-w · Cited by 44 · cited 1×
“Such estimates would be concordant with the trend previously identified in the field of mendelian randomization, another area which has seen a tsunami of formulaic manuscripts [26].”
Erasmus, A., Holman, B., & Ioannidis, J. P. A. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111584 · Cited by 41
A call for research to address the threat of paper mills
Byrne, J. A., Abalkina, A., Akinduro-Aje, O., Christopher, J., Eaton, S. E., Joshi, N., Scheffler, U., Wise, N. H., & Wright, J. (2024) · PLOS Biology · doi:10.1371/journal.pbio.3002931 · Cited by 38 · cited 2×
“These are all key questions and issues that have recently been highlighted by the United2Act Research Group and others [1,27,28], as well as being part of the wider debate around AI usage in research by bodies including the European Commission [29].”
OpenSAFELY: A platform for analysing electronic health records designed for reproducible research
Nab, L., Schaffer, A. L., Hulme, W., DeVito, N. J., Dillingham, I., Wiedemann, M., Andrews, C. D., Curtis, H., Fisher, L., Green, A., et al. (2024) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5815 · Cited by 34 · cited 1×
“It would also mitigate the risks of data dredging by paper mills, in-line with similar approaches such as pre-registration of studies or models such as OpenSafely employed by the UK’s National Health Service [35].”
AI-readiness Criteria for Biomedical Data
Clark, T., Caufield, H., Parker, J. A., Al Manir, S., Amorim, E., Eddy, J., Gim, N., Gow, B., Goar, W., Hansen, J. N., et al. (2024) · doi:10.1101/2024.10.23.619844 · Cited by 32
Multiple analyses in clinical trials: sound science or data dredging?
Lord, S. J., Gebski, V., & KEECH, A. C. (2004) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2004.tb06376.x · Cited by 29 · cited 1×
“While data dredging is a well-described phenomenon [13–16], direct-to-AI pipelines can make formulaic research pipelines more productive than has previously been possible.”
Multifactorial prediction of depression diagnosis and symptom dimensions
McNamara, M. E., Shumake, J., Stewart, R. A., Labrada, J., Alario, A., Allen, J. J. B., Palmer, R., Schnyer, D. M., McGeary, J. E., & Beevers, C. G. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.113805 · Cited by 23
Research misconduct in China: towards an institutional analysis
Zhang, X., & Wang, P. (2024) · Research Ethics · doi:10.1177/17470161241247720 · Cited by 21 · cited 1×
“While this may be a product of different incentives in China leading to misconduct [18,19], it may also reflect the ability of small networks or groups of individuals to exploit AI-ready datasets for end-to-end generation of very large numbers of manuscripts.”
Post-publication Peer Review: PubPeer
Townsend, F. (2013) · Editors Bulletin · doi:10.1080/17521742.2013.865333 · Cited by 14 · cited 1×
“The scientific community at large can also contribute through proactive follow-up commentary using post-publication fora such as PubPeer [37].”
Ortega, J. L., & Delgado‐Quirós, L. (2023) · El Profesional de la Informacion · doi:10.3145/epi.2023.ene.18 · Cited by 14
The bridge between lab and clinic
Wadman, M. (2010) · Nature · doi:10.1038/468877a · Cited by 12 · cited 1×
“Best practice is to conduct research in a comprehensive manner, or run the risk of research not being translational [23], with real costs to healthcare systems and patients [24,25].”
Data dredging and false discovery
Elston, D. M. (2019) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2019.07.061 · Cited by 9
Artificial intelligence – the Janus-faced tool in our hands
Netzer, N. C. (2024) · Sleep And Breathing · doi:10.1007/s11325-024-03129-7 · Cited by 6
Chen, C.-Y., Chen, J., Ree, S., Chang, C., & Yu, S.-H. (2020) · Annals of General Psychiatry · doi:10.1186/s12991-020-00315-1 · Cited by 5
“Houston We Have a Problem….”: Raising the Quality and Authenticity of Manuscripts Submitted to JCEM
Stewart, P. M. (2025) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgaf036 · Cited by 5 · cited 1×
“While data dredging is a well-described phenomenon [13–16], direct-to-AI pipelines can make formulaic research pipelines more productive than has previously been possible.”
Observations from Statistical Review Editors: A Commentary
Spick, M. P., Higgins, J., Green, C. L., Matsouaka, R., Shin, D. B., Hall, R. P., & Geifman, N. (2024) · JID Innovations · doi:10.1016/j.xjidi.2024.100302 · Cited by 3 · cited 1×
“Publishers should also give consideration to appointing dedicated statistical reviewers, a practice already being adopted by some journals [36].”
Self-citationThe potential burden of overinvestigation
Dobson, J. (2024) · BMJ · doi:10.1136/bmj.q1748 · Cited by 2 · cited 1×
“Best practice is to conduct research in a comprehensive manner, or run the risk of research not being translational [23], with real costs to healthcare systems and patients [24,25].”
Research works printed but not linked: 1
- About COPE.
Cited by
Generative artificial intelligence in medicine
Teo, Z. L., Thirunavukarasu, A. J., Elangovan, K., Cheng, H., Moova, P., Soetikno, B., Nielsen, C., Pollreisz, A., Ting, D. S. J., Morris, R. J. T., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03983-2 · Cited by 119
“…of this automation will be positive: a dramatic increase in formulaic reports of studies analyzing publicly available datasets has been observed since the proliferation of GAI chatbots, with many of these studies being of poor quality and likely originating from paper mills and citation farms 118 .”
Rudan, I., Song, P., Adeloye, D., & Campbell, H. (2025) · Journal of Global Health · doi:10.7189/jogh.15.01004 · Cited by 75
Spick, M. P., Onoja, A., Harrison, C., Stender, S., Byrne, J. A., & Geifman, N. (2026) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2026.112203 · Cited by 48
Self-citationThe Paradox of Ethical AI-Assisted Research
Mezzadri, D. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-025-09671-7 · Cited by 31
“…falsehoods, mistakes, fabrications, and biased content (Chauhan & Currie, 2024; Cheng et al., 2025; Elali & Rachid, 2023; Prilla-man, 2024; Resnik et al., 2025; Rodriguez-Saavedra et al., 2025; Suchak et al., 2025; Walters & Wilder, 2023 ), thus lowering the quality of academic publishing.”
An atlas of exposome–phenome associations in health and disease risk
Patel, C. J., Ioannidis, J. P. A., & Manrai, A. K. (2026) · Nature Medicine · doi:10.1038/s41591-026-04266-0 · Cited by 32
“These candidate studies are presented selectively in millions of papers on claimed associations yielding fragmented and often biased snapshots of the exposome–phenotype maze 6 .”
Metadata
- Authors
- Suchak, T., Aliu, A. E., Harrison, C., Zwiggelaar, R., Geifman, N., & Spick, M.
- Affiliations
- University of Surrey (GB)
- Aberystwyth University (GB)
- Corresponding author
- Spick, Matt
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2025-05-08
- Volume
- 23
- Issue
- 5
- Pages
- e3003152
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC12061153
- OpenAlex
- W4410206242
- Cited by
- 95
- References
- 30
- Citations by year
- 2026: 54 · 2025: 41
- FWCI
- 98.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Biotechnology and Biological Sciences Research Council (BB/Y006933/1)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- paper mills, single factor analysis, NHANES, artificial intelligence in research, p-hacking, false discovery, multifactorial analysis, research quality, statistical practice, peer review
- MeSH terms
- Artificial Intelligence, Humans, Nutrition Surveys, Research Design, United States, Databases, Factual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.