Uniform consistency in causal inference
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
There is a long tradition of representing causal relationships by directed acyclic graphs (Wright, 1934). Spirtes (1994), Spirtes et al. (1993) and Pearl & Verma (1991) describe procedures for inferring the presence or absence of causal arrows in the graph even if there might be unobserved confounding variables, and/or an unknown time order, and that under weak conditions, for certain combinations of directed acyclic graphs and probability distributions, are asymptotically, in sample size, consistent. These results are surprising since they seem to contradict the standard statistical wisdom that consistent estimators of causal effects do not exist for nonrandomised studies if there are potentially unobserved confounding variables. We resolve the apparent incompatibility of these views by closely examining the asymptotic properties of these causal inference procedures. We show that the asymptotically consistent procedures are ‘pointwise consistent’, but ‘uniformly consistent’ tests do not exist. Thus, no finite sample size can ever be guaranteed to approximate the asymptotic results. We also show the nonexistence of valid, consistent confidence intervals for causal effects and the nonexistence of uniformly consistent point estimators. Our results make no assumption about the form of the tests or estimators. In particular, the tests could be classical independence tests, they could be Bayes tests or they could be tests based on scoring methods such as BIC or AIC. The implications of our results for observational studies are controversial and are discussed briefly in the last section of the paper. The results hinge on the following fact: it is possible to find, for each sample size n, distributions P and Q such that P and Q are empirically indistinguishable and yet P and Q correspond to different causal effects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
Robins, J. (1986) · Mathematical Modelling · doi:10.1016/0270-0255(86)90088-6 · Cited by 2,451
Self-citationCausal diagrams for empirical research
PEARL, J. (1995) · Biometrika · doi:10.1093/biomet/82.4.669 · Cited by 2,355 · cited 1×
“The model is closely related to Wright’s path diagrams (Wright, 1934) and structural equation models with acyclic path diagrams and uncorrelated errors (Bollen, 1989, pp. 80–2; Spirtes et al., 2000, pp. 27–8; Pearl, 1995).”
MethodBounds on Treatment Effects from Studies with Imperfect Compliance
Balke, A., & Pearl, J. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474074 · Cited by 564
Causal Inference from Complex Longitudinal Data
Robins, J. M. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-1842-5_4 · Cited by 510
Self-citationStrong Completeness and Faithfulness in Bayesian Networks
Meek, C. (2013) · arXiv · doi:10.48550/arxiv.1302.4973 · Cited by 194
Cited by
Detecting and quantifying causal associations in large nonlinear time series datasets
Runge, J., Nowack, P., Kretschmer, M., Flaxman, S., & Sejdinovic, D. (2019) · Science Advances · doi:10.1126/sciadv.aau4996 · Cited by 979
Estimating high-dimensional directed acyclic graphs with the PC-algorithm
Kalisch, M., & Bühlmann, P. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.math/0510436 · Cited by 591
Optimal Structural Nested Models for Optimal Sequential Decisions
Robins, J. M. (2004) · Lecture Notes in Statistics · doi:10.1007/978-1-4419-9076-1_11 · Cited by 569
Self-citationCausal network reconstruction from time series: From theoretical assumptions to practical estimation
Runge, J. (2018) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5025050 · Cited by 422
High-dimensional variable selection
Wasserman, L., & Roeder, K. (2009) · The Annals of Statistics · doi:10.1214/08-aos646 · Cited by 373
Metadata
- Authors
- Robins, J. M.
- Venue
- Biometrika
- Publisher
- Oxford University Press (OUP)
- Published
- 2003-09-01
- Volume
- 90
- Issue
- 3
- Pages
- 491-515
- ISSN
- 0006-3444 (ISSN-L)
- 1464-3510
- OpenAlex
- W1964992876
- Cited by
- 136
- References
- 7
- FWCI
- 4.24
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.