Explosion of formulaic research articles, including inappropriate study designs and false discoveries, based on the NHANES US national health database
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
With the growth of artificial intelligence (AI)-ready datasets such as the National Health and Nutrition Examination Survey (NHANES), new opportunities for data-driven research are being created, but also generating risks of data exploitation by paper mills. In this work, we focus on two areas of potential concern for AI-supported research efforts. First, we describe the production of large numbers of formulaic single-factor analyses, relating single predictors to specific health conditions, where multifactorial approaches would be more appropriate. Employing AI-supported single-factor approaches removes context from research, fails to capture interactions, avoids false discovery correction, and is an approach that can easily be adopted by paper mills. Second, we identify risks of selective data usage, such as analyzing limited date ranges or cohort subsets without clear justification, suggestive of data dredging, and post-hoc hypothesis formation. Using a systematic literature search for single-factor analyses, we identified 341 NHANES-derived research papers published over the past decade, each proposing an association between a predictor and a health condition from the wide range contained within NHANES. We found evidence that research failed to take account of multifactorial relationships, that manuscripts did not account for the risks of false discoveries, and that researchers selectively extracted data from NHANES rather than utilizing the full range of data available. Given the explosion of AI-assisted productivity in published manuscripts (the systematic search strategy used here identified an average of 4 papers per annum from 2014 to 2021, but 190 in 2024–9 October alone), we highlight a set of best practices to address these concerns, aimed at researchers, data controllers, publishers, and peer reviewers, to encourage improved statistical practices and mitigate the risks of paper mills using AI-assisted workflows to introduce low-quality manuscripts to the scientific literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A recent investigation revealed systematic fabrication of research manuscripts using NHANES data, resulting in hundreds of retractions [6].”
From Citation Hallucination Determines Success: An Empirical Comparison of Six Medical AI Research Systems (2026) · cites this paper as [6]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · cited 1×
“In addition, the ability to generate large numbers of machine-learning models allows for rapid post-hoc investigation of alternative hypotheses, should the main a priori hypothesis not be supported (a form of hypothesizing after the results are known, or HARKing) [10–12].”
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · cited 2×
“It is not possible to know which are examples of hypothesizing after the results are known (HARKing) [10,11], but the lack of disclosure and justification of choosing subsets is clearly a shortfall versus best practice.”
Atlas of plasma NMR biomarkers for health and disease in 118,461 individuals from the UK Biobank
Julkunen, H., Cichońska, A., Tiainen, M., Koskela, H., Nybo, K., Mäkelä, V., Nokso-Koivisto, J., Kristiansson, K., Perola, M., Salomaa, V., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-36231-7 · Cited by 536 · cited 2×
“This has previously been demonstrated in UK Biobank data, where latent systemic connectivity was found in common biomarkers across a wide range of health conditions [9].”
Ten things to know about ten cardiovascular disease risk factors
Bays, H. E., Taub, P. R., Epstein, E., Michos, E. D., Ferraro, R. A., Bailey, A. L., Kelli, H. M., Ferdinand, K. C., Echols, M. R., Weintraub, H., et al. (2021) · American Journal of Preventive Cardiology · doi:10.1016/j.ajpc.2021.100149 · Cited by 274
The fight against fake-paper factories that churn out sham science
Else, H., & Van Noorden, R. (2021) · Nature · doi:10.1038/d41586-021-00733-5 · Cited by 251 · cited 1×
“While this may be a product of different incentives in China leading to misconduct [18,19], it may also reflect the ability of small networks or groups of individuals to exploit AI-ready datasets for end-to-end generation of very large numbers of manuscripts.”
Rubin, M. (2017) · Review of General Psychology · doi:10.1037/gpr0000128 · Cited by 182 · cited 1×
“It is not possible to know which are examples of hypothesizing after the results are known (HARKing) [10,11], but the lack of disclosure and justification of choosing subsets is clearly a shortfall versus best practice.”
Byrne, J. A., & Christopher, J. (2020) · FEBS Letters · doi:10.1002/1873-3468.13747 · Cited by 133
Data-Dredging Procedures in Survey Analysis
Selvin, H., & Stuart, A. (1966) · The American Statistician · doi:10.1080/00031305.1966.10480401 · Cited by 132
Alberts, B., Hanson, B., & Kelner, K. L. (2008) · Science · doi:10.1126/science.1162115 · Cited by 119 · cited 1×
“…such as Committee on Publication Ethics (COPE) provide support and guidance to editors, publishers, and researchers [30], often peer reviewers will be the last, and therefore the most important, line of defence against poor practices [31], including those that may be adopted by paper mills.”
Research works printed but not linked: 1
- About COPE.
Cited by
Items 101–113 of 113
AI Writing Vocabulary in Ophthalmology: Faster Non-Native Adoption, Unchanged Outcomes
Berni, A., Ting, D. S. W., Caputo, G., Russo, A., Gutiérrez-Sinisterra, L., Avitabile, A., Virgili, G., Reibaldi, M., Giansanti, F., Bacherini, D., et al. (2026) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2026.08.011
Tips for Writing a Successful Research Article
Duell, P. B., & Goldberg, A. C. (2026) · Journal of clinical lipidology · doi:10.1016/j.jacl.2026.07.039
Zhang, H., Rodemade, A., Wang, S., Zhang, X., Lee, S. Y., Evangelopoulos, D., Tian, M., & Wang, Z. (2026) · BMJ Public Health · doi:10.1136/bmjph-2025-004782
“Future research should prioritise expanding the geographic scope of studies focusing on continents other than Europe and Asia, alternative air pollutants other than PM 2.5 and NO 2 , rural areas, older population groups, HAP measurements and polypharmacy.”
InfluentialSalman, B. M., Mohamed, A., Shaikh, A. J., Mahrizi, A. A., Shidi, R. A., Farsi, S. A., Hashish, N., AL-Kiyumi, Z., Ayaad, O., & Al-Baimani, K. S. (2026) · Exploratory Research in Clinical and Social Pharmacy · doi:10.1016/j.rcsop.2026.100852
“26 These issues are particularly important in PPS, where AI tools may influence medication-related decisions, patient safety, and regulatory processes.”
Old data, new problem: rethinking public database research in the era of generative AI
You, Y., Zheng, K., Ablitip, A., Ding, H., & Chen, Y. (2026) · Medicine Discovery · doi:10.1016/j.medd.2026.100002
Coping with the impacts of generative artificial intelligence on scholarly publications
Hosseini, M., Rasmussen, L., & Resnik, D. B. (2026) · Accountability in Research · doi:10.1080/08989621.2026.2737718
Scientific Writing, Peer Review, and Artificial Intelligence
BaHammam, A. S. (2026) · Sleep Medicine Clinics · doi:10.1016/j.jsmc.2026.08.004
Restructuring Scientific Papers for Human and AI Readers
Lin, Z. (2026) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459261480685 · Cited by 1
Cheap Production, Costly Verification: Doing Research under Agentic AI and AutomationPreprint
Nguyen, M. A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7631798
Verification Debt as Agentic Artificial Intelligence Accelerates Clinical Analysis (Preprint)Preprint
Guillen-Ramirez, H., Endrich, O., & Beldi, G. (2026) · doi:10.2196/preprints.114328
Addressing threats to validity in large dataset research
Yoo, W., & Ioannidis, J. P. A. (2026) · PLoS Medicine · doi:10.1371/journal.pmed.1005282
Bribiescas, R. G. (2026) · American Journal of Human Biology · doi:10.1002/ajhb.70358
Gibson, A. D., Borg, D. N., White, N. M., & Barnett, A. G. (2026) · medRxiv · doi:10.64898/2026.11.02.26367135
Metadata
- Authors
- Suchak, T., Aliu, A. E., Harrison, C., Zwiggelaar, R., Geifman, N., & Spick, M.
- Affiliations
- University of Surrey (GB)
- Aberystwyth University (GB)
- Corresponding author
- Spick, Matt
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2025-05-08
- Volume
- 23
- Issue
- 5
- Pages
- e3003152
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC12061153
- OpenAlex
- W4410206242
- Cited by
- 95
- References
- 30
- Citations by year
- 2026: 54 · 2025: 41
- FWCI
- 98.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Biotechnology and Biological Sciences Research Council (BB/Y006933/1)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- paper mills, single factor analysis, NHANES, artificial intelligence in research, p-hacking, false discovery, multifactorial analysis, research quality, statistical practice, peer review
- MeSH terms
- Artificial Intelligence, Humans, Nutrition Surveys, Research Design, United States, Databases, Factual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.