Random Allocation in Observational Data: How Small But Robust Effects Could Facilitate Hypothesis-free Causal Inference
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Conventional observational epidemiology has an unenviable reputation for generating false-positive findings,1,2 or “scares,” as others call them.3 In 1993, for example, the New York Times reported that “vitamin E greatly reduces the risk of heart disease”4 following simultaneous publication of 2 observational studies in the New England Journal of Medicine demonstrating that use of vitamin E supplements, even for just a few years, was associated with a substantially lower risk of coronary heart disease (CHD).5,6 Randomized controlled trials (RCTs)—testing precisely the same hypothesis—revealed no reduction in risk at all.7 As I write this, UK news media are reporting that “junk food makes your kids dumb,” in response to a paper reporting that children given a poor diet at age 3 had lower IQ scores 5 years later.8 In the latter case, however, no RCTs are ever likely to be carried out, and the status of the finding will remain liminal. The ratio of false-positive to false-negative (FP:FN) published findings in traditional epidemiology is very high, John Ioannidis and colleagues argue.9 But what exactly are “false-positive” findings in the context of conventional observational epidemiology, and how can their prevalence be quantified? My suspicion is that use of vitamin E supplements would—in most contexts—be associated with lower risk of CHD, because of substantial confounding by a myriad of measured and unmeasured factors related to such a socially and behaviorally patterned exposure.10 The same will apply to children fed on junk food and their later intelligence.11 The inability to substantially “control” statistically for confounding in many situations—due to measurement error in assessed confounders and omission of others—remains underappreciated.12,13 In the sense that these associations do exist, they should perhaps not be called false positives; they are false positives only if they are taken to be indicators of underlying causal effects. Noncausal but replicable observational associations could clearly be of considerable value through prediction of disease risk, and targeting of preventive measures to those who can benefit most. For the holy grail of epidemiology—identifying causes of disease—they are a disappointment, however. A wide range of approaches—from formally comparing associations in contexts where confounding structures differ,14 utilizing correlates of the exposure under study that are not plausible causes15 and natural experiments,16 through to formal instrumental variables methods17—offer greater hope for reliable causal inference than plowing on with traditional approaches and keeping the FP:FN ratio high.9 Ioannidis and colleagues reiterate the low positive predictive value of a nominally “significant” P, something I (along with a generation of epidemiologists, I imagine) first encountered in Michael Oakes' seminal “Statistical Inference,18 although (as has been publically confessed) it did not instantly prevent me from using the word ‘significant’ and over-interpreting such tests.19As Sander Greenland20 argued in this journal 20 years ago, randomization provides the key link between inferential statistics and causal parameters.” It is through this prism that we should consider Ioannidis' finding that in genetic association studies, once appropriate thresholds were applied,21 false positives became a rarity.9 Greenland stated that his arguments were “largely derived from the writings of R.A. Fisher,”20 and it was Fisher who clarified that randomization is inherent in genetic analysis. When lecturing on “Statistical Methods in Genetics” in 1951, Fisher clarified the relationship between the 2 disciplines to which he contributed so much22: And here I may mention a connection between our two subjects which seem not to be altogether accidental, namely that the factorial method of experimentation, now of lively concern so far afield as the psychologists, or the industrial chemists, derives its structure and its name, from the simultaneous inheritance of Mendelian factors. Geneticists certainly need not feel that the intellectual debt is all on one side.22 Fisher goes on: Genetics is indeed in a peculiarly favoured condition in that Providence has shielded the geneticist from many of the difficulties of a reliably controlled comparison. The different genotypes possible from the same mating have been beautifully randomised by the meiotic process.22 This principle—that analysis of genetic data is analogous to that of a randomized experiment—has, in epidemiology, been termed “Mendelian randomization.”23 This depends on the basic (but approximate) laws of Mendelian genetics. If the probability that a postmeiotic germ cell that has received any particular allele at segregation contributes to a viable conceptus is independent of environment (following from Mendel's first law), and if genetic variants sort independently (following on from Mendel's second law), then these variants will not be associat
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
XV.—The Correlation between Relatives on the Supposition of Mendelian Inheritance.
Fisher, R. A. (1919) · Transactions of the Royal Society of Edinburgh · doi:10.1017/s0080456800012163 · Cited by 4,387
Law, M. R., Morris, J. K., & Wald, N. J. (2009) · BMJ · doi:10.1136/bmj.b1665 · Cited by 2,775
Sterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466
Smith, G. D., Lawlor, D. A., Harbord, R., Timpson, N., Day, I., & Ebrahim, S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040352 · Cited by 569 · cited 1×
“to nongenetic confounding factors, as has been empirically demonstrated.(25)”
BackgroundSelf-citationLawlor, D. A., Smith, G. D., Bruckdorfer, K. R., Kundu, D., & Ebrahim, S. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16260-0 · Cited by 438
Self-citationRandomization, Statistics, and Causal Inference
Greenland, S. (1990) · Epidemiology · doi:10.1097/00001648-199011000-00003 · Cited by 385
The False-positive to False-negative Ratio in Epidemiologic Studies
Ioannidis, J. P. A., Tarone, R., & McLaughlin, J. K. (2011) · Epidemiology · doi:10.1097/ede.0b013e31821b506e · Cited by 321
Phillips, A. N., & Smith, G. D. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90155-3 · Cited by 275 · cited 1×
“The inability to substantially “control” statistically for confounding in many situations—due to measurement error in assessed confounders and omission of others—remains underappreciated.(12,13) In the sense that these associations do exist, they should perhaps not be called false positives; they are false positives o…”
BackgroundSelf-citationSmith, G. D. (2008) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/j.1742-7843.2007.00191.x · Cited by 257
Self-citation
Cited by
Bowden, J., Davey Smith, G., & Burgess, S. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv080 · Cited by 9,816
“It is also consistent if the pleiotropic effects happen to cancel out, such that the bias term is equal to zero.(24) Although this will not be universally plausible, we explore the condition that the correlation between the genetic associations with the exposure (the cj parameters) and the direct effects of the geneti…”
BackgroundMendelian randomization: genetic anchors for causal inference in epidemiological studies
Davey Smith, G., & Hemani, G. (2014) · Human Molecular Genetics · doi:10.1093/hmg/ddu328 · Cited by 4,677
Burgess, S., Bowden, J., Fall, T., Ingelsson, E., & Thompson, S. G. (2017) · Epidemiology · doi:10.1097/ede.0000000000000559 · Cited by 2,017
Burgess, S., & Thompson, S. G. (2015) · American Journal of Epidemiology · doi:10.1093/aje/kwu283 · Cited by 1,605
Recent Developments in Mendelian Randomization Studies
Zheng, J., Baird, D., Borges, M.-C., Bowden, J., Hemani, G., Haycock, P., Evans, D. M., & Smith, G. D. (2017) · Current Epidemiology Reports · doi:10.1007/s40471-017-0128-6 · Cited by 1,147
Metadata
- Authors
- Davey Smith, G.
- Venue
- Epidemiology
- Published
- 2011-07
- Volume
- 22
- Issue
- 4
- Pages
- 460-463
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2324387650
- Cited by
- 48
- References
- 12
- FWCI
- 4.73
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Random Allocation in Observational Data: How Small But Robust Effects Could Facilitate Hypothesis-free Causal Inference. https://app.cochat.ai/papers/10.1097/ede.0b013e31821d0426
- The paper
- Davey Smith, G. (2011). Random Allocation in Observational Data: How Small But Robust Effects Could Facilitate Hypothesis-free Causal Inference. Epidemiology. https://doi.org/10.1097/ede.0b013e31821d0426
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.