Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
A large majority of human nutrition research uses nonrandomized observational designs, but this has led to little reliable progress. This is mostly due to many epistemologic problems, the most important of which are as follows: difficulty detecting small (or even tiny) effect sizes reliably for nutritional risk factors and nutrition-related interventions; difficulty properly accounting for massive confounding among many nutrients, clinical outcomes, and other variables; difficulty measuring diet accurately; and suboptimal research reporting. Tiny effect sizes and massive confounding are largely unfixable problems that narrowly confine the scenarios in which nonrandomized observational research is useful. Although nonrandomized studies and randomized trials have different priorities (assessment of long-term causality compared with assessment of treatment effects), the odds for obtaining reliable information with the former are limited. Randomized study designs should therefore largely replace nonrandomized studies in human nutrition research going forward. To achieve this, many of the limitations that have traditionally plagued most randomized trials in nutrition, such as small sample size, short length of follow-up, high cost, and selective reporting, among others, must be overcome. Pivotal megatrials with tens of thousands of participants and lifelong follow-up are possible in nutrition science with proper streamlining of operational costs. Fixable problems that have undermined observational research, such as dietary measurement error and selective reporting, need to be addressed in randomized trials. For focused questions in which dietary adherence is important to maximize, trials with direct observation of participants in experimental in-house settings may offer clean answers on short-term metabolic outcomes. Other study designs of randomized trials to consider in nutrition include registry-based designs and "N-of-1" designs. Mendelian randomization designs may also offer some more reliable leads for testing interventions in trials. Collectively, an improved randomized agenda may clarify many things in nutrition science that might never be answered credibly with nonrandomized observational designs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
2 retracted
RETRACTED: Primary Prevention of Cardiovascular Disease with a Mediterranean Diet
Estruch, R., Ros, E., Salas-Salvadó, J., Covas, M.-I., Corella, D., Arós, F., Gómez-Gracia, E., Ruiz-Gutiérrez, V., Fiol, M., Lapetra, J., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1200303 · Cited by 4,257
RetractedEstruch, R., Ros, E., Salas-Salvadó, J., Covas, M.-I., Corella, D., Arós, F., Gómez-Gracia, E., Ruiz-Gutiérrez, V., Fiol, M., Lapetra, J., et al. (2018) · New England Journal of Medicine · doi:10.1056/nejmc1806491 · Cited by 1,008
RetractedGRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“significant result represents a true effect (53).”
Self-citation(1994) · New England Journal of Medicine · doi:10.1056/nejm199404143301501 · Cited by 4,737
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Self-citationThe PRECIS-2 tool: designing trials that are fit for purpose
Loudon, K., Treweek, S., Sullivan, F., Donnan, P., Thorpe, K. E., & Zwarenstein, M. (2015) · BMJ · doi:10.1136/bmj.h2147 · Cited by 1,846 · cited 1×
“The megatrials should reflect pragmatic circumstances so that their results can be readily translated to recommendations for the general public (54).”
Surrogate End Points in Clinical Trials: Are We Being Misled?
Fleming, T. R., & DeMets, D. L. (1996) · Annals of Internal Medicine · doi:10.7326/0003-4819-125-7-199610010-00011 · Cited by 1,651
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Self-citationMente, A., de Koning, L., Shannon, H. S., & Anand, S. S. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.38 · Cited by 1,267
Mendelian randomization: prospects, potentials, and limitations
Smith, G. D. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh132 · Cited by 1,222
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127 · cited 1×
“It is alsomore vulnerable to publication and selective-outcome reporting biases (47) compared with randomized research, because it is more difficult to ascertain how many observational data sets are available worldwide that can address a given exposure-outcome relation (48).”
Gorber, S. C., Schofield-Hurwitz, S., Hardt, J., Levasseur, G., & Tremblay, M. (2009) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntn010 · Cited by 1,079 · cited 1×
“Simply adjusting for smoking in the regression model probably will not suffice, because several additional variables may need to be accounted for, and tobacco exposure and nutrient intake are both measured with considerable error (12, 36).”
Bjelakovic, G., Nikolova, D., Gluud, L. L., Simonetti, R. G., & Gluud, C. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007176.pub2 · Cited by 924
Red and Processed Meat and Colorectal Cancer Incidence: Meta-Analysis of Prospective Studies
Chan, D. S. M., Lau, R., Aune, D., Vieira, R., Greenwood, D. C., Kampman, E., & Norat, T. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020456 · Cited by 903 · cited 1×
“18/50 g processedmeat (22) is too small to avoid residual uncertainty given the other problems that we discuss below.”
Revised standards for statistical evidence
Johnson, V. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313476110 · Cited by 808
The Impact of Residual and Unmeasured Confounding in Epidemiologic Studies: A Simulation Study
Fewell, Z., Davey Smith, G., & Sterne, J. A. C. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwm165 · Cited by 734
The Randomized Registry Trial — The Next Disruptive Technology in Clinical Research?
Lauer, M. S., & D'Agostino, R. B. (2013) · New England Journal of Medicine · doi:10.1056/nejmp1310102 · Cited by 595 · cited 1×
“Additional cost-savings will be achieved by streamlining the trial design (55) and, when possible, by building the trial on the platform of an already-existing health registry (56).”
A "SMART" Design for Building Individualized Treatment Sequences
Lei, H., Nahum-Shani, I., Lynch, K., Oslin, D., & Murphy, S. A. (2012) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032511-143152 · Cited by 557 · cited 1×
“Investigations of this sort can first be evaluated in Sequential Multiple Assignment RandomizedTrials to develop an adaptive intervention, which can then be evaluated in a randomized confirmatory trial against an appropriate alternative (68).”
Evolution of Reporting P Values in the Biomedical Literature, 1990-2015
Chavalarias, D., Wallach, J. D., Li, A. H. T., & Ioannidis, J. P. A. (2016) · JAMA · doi:10.1001/jama.2016.1952 · Cited by 370
Self-citationSchwingshackl, L., Knüppel, S., Schwedhelm, C., Hoffmann, G., Missbach, B., Stelmach-Mardas, M., Dietrich, S., Eichelmann, F., Kontopanteils, E., Iqbal, K., et al. (2016) · Advances in Nutrition · doi:10.3945/an.116.013052 · Cited by 360
Registry-based randomized clinical trials—a new clinical trial paradigm
James, S., Rao, S. V., & Granger, C. B. (2015) · Nature Reviews Cardiology · doi:10.1038/nrcardio.2015.33 · Cited by 331
Hall, K. D., Chen, K. Y., Guo, J., Lam, Y. Y., Leibel, R. L., Mayer, L. E., Reitman, M. L., Rosenbaum, M., Smith, S. R., Walsh, B. T., et al. (2016) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.116.133561 · Cited by 330 · cited 1×
“For these questions, trials can be performed that involve continuous direct observation of participants in in-house settings (87, 88).”
Miller, P. E., & Perez, V. (2014) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.113.082826 · Cited by 313
Hall, K. D., Bemis, T., Brychta, R., Chen, K. Y., Courville, A., Crayner, E. J., Goodwin, S., Guo, J., Howard, L., Knuth, N. D., et al. (2015) · Cell Metabolism · doi:10.1016/j.cmet.2015.07.021 · Cited by 280 · cited 1×
“For these questions, trials can be performed that involve continuous direct observation of participants in in-house settings (87, 88).”
Ye, Y., Li, J., & Yuan, Z. (2013) · PLoS ONE · doi:10.1371/journal.pone.0056803 · Cited by 204
Bjelakovic, G., Nikolova, D., & Gluud, C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0074558 · Cited by 183
Subgroup analyses in therapeutic cardiovascular clinical trials: Are most of them misleading?
Hernández, A. V., Boersma, E., Murray, G. D., Habbema, J. D. F., & Steyerberg, E. W. (2006) · American Heart Journal · doi:10.1016/j.ahj.2005.04.020 · Cited by 131 · cited 1×
“Rarely are formal interaction tests performed to evaluate this specifically (61), and even more rarely are claims of interaction shown to be credible (62, 63).”
Making Prospective Registration of Observational Research a Reality
Dal-Ré, R., Ioannidis, J. P., Bracken, M. B., Buffler, P. A., Chan, A.-W., Franco, E. L., La Vecchia, C., & Weiderpass, E. (2014) · Science Translational Medicine · doi:10.1126/scitranslmed.3007513 · Cited by 123
Self-citationStraube, B. (2012) · Behavioral and Brain Functions · doi:10.1186/1744-9081-8-35 · Cited by 96
Risk factors and interventions with statistically significant tiny effects
Siontis, G. C., & Ioannidis, J. P. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr099 · Cited by 80
Self-citationWhy do we think we know what we know? A metaknowledge analysis of the salt controversy
Trinquart, L., Johns, D. M., & Galea, S. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyv184 · Cited by 77 · cited 2×
“nonrandomized observational data has created widespread confusion about optimal nutrition (1, 2).”
Unscientific Beliefs about Scientific Topics in Nutrition
Brown, A. W., Ioannidis, J. P. A., Cope, M. B., Bier, D. M., & Allison, D. B. (2014) · Advances in Nutrition · doi:10.3945/an.114.006577 · Cited by 54
Self-citationRobust empirical calibration of p‐values using observational data
Schuemie, M. J., Hripcsak, G., Ryan, P. B., Madigan, D., & Suchard, M. A. (2016) · Statistics in Medicine · doi:10.1002/sim.6977 · Cited by 53
Cited by
The Challenge of Reforming Nutritional Epidemiologic Research
Ioannidis, J. P. A. (2018) · JAMA · doi:10.1001/jama.2018.11025 · Cited by 389
Self-citationPaving the Way to Precision Nutrition Through Metabolomics
Tebani, A., & Bekri, S. (2019) · Frontiers in Nutrition · doi:10.3389/fnut.2019.00041 · Cited by 143
Orsso, C. E., Montes‐Ibarra, M., Findlay, M., van der Meij, B. S., de van der Schueren, M. A. E., Landi, F., Laviano, A., & Prado, C. M. (2022) · Journal of Cachexia, Sarcopenia and Muscle · doi:10.1002/jcsm.12954 · Cited by 73
Pan, B., Ge, L., Lai, H., Wang, Q., Wang, Q., Zhang, Q., Yin, M., Li, S., Tian, J., Yang, K., et al. (2022) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2021.1937040 · Cited by 68
The harms of high protein intake: conjectured, postulated, claimed, and presumed, but shown?
French, S. J., Kanter, M., Maki, K. C., Rust, B. M., & Allison, D. B. (2025) · The American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2025.05.002 · Cited by 15
Metadata
- Authors
- Trepanowski, J. F., & Ioannidis, J. P. A.
- Venue
- Advances in Nutrition
- Publisher
- Elsevier BV
- Published
- 2018-07
- Volume
- 9
- Issue
- 4
- Pages
- 367-377
- ISSN
- 2156-5376 (ISSN-L)
- 2161-8313
- OpenAlex
- W2883885422
- Cited by
- 105
- References
- 34
- FWCI
- 9.88
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.