The DOI matches its Crossref record; 60 of the 62 references it cites were found in a registry or an index.
Statistics and Causal Inference
Abstract
Problems involving causal inference have dogged at the heels of statistics since its earliest days. Correlation does not imply causation, and yet causal conclusions drawn from a carefully designed experiment are often valid. What can a statistical model say about causation? This question is addressed by using a particular model for causal inference (Holland and Rubin 1983; Rubin 1974) to critique the discussions of other writers on causation and causal inference. These include selected philosophers, medical researchers, statisticians, econometricians, and proponents of causal modeling.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As is usually the case, due to the fundamental problem of causal inference (Holland, 1986), we only observe one of the two potential outcomes for each unit i .”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Holland, 1986
In the citing paper
“causal effects can never be observed directly”
From Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable (2014) · cites this paper as Holland 1986
In the citing paper
“In uplift modeling, however, the actual outcome that is estimated is unobserved, and therefore the error made by the model cannot be measured at the individual customer level.”
From Why you should stop predicting customer churn and start using uplift models (2021) · cites this paper as [34]
In the citing paper
“Historical data containing information on observed customer behavior never allows classifying customers in the four groups shown in Table 1, as the net e ↵ ect of treatment cannot be observed for an individual customer due to the fundamental problem of causal inference [34].”
From Why you should stop predicting customer churn and start using uplift models (2021) · cites this paper as [34]
In the citing paper
“Uplift or, in other words, whether a customer is persuadable, a lost cause, do-notdisturb or a sure thing, cannot be observed due to the fundamental problem of causal inference [34].”
From Why you should stop predicting customer churn and start using uplift models (2021) · cites this paper as [34]
In the citing paper
“Since the antecedent variable is an experimental variable in a randomized study, its causal relation to the mediator and the outcome variable can be interpreted unambiguously102,103.”
From Near transfer to an unrelated N-back task mediates the effect of N-back working memory training on matrix reasoning (2022) · cites this paper as [102]
In the citing paper
“Key to the ability to make inferences about mean causal effects are the assumptions of unit homogeneity and independence of units (Holland, 1986; King, Keohane and Verba, 1994: 91-97).”
From Multi-Method Research in the Social Sciences: A Review of Recent Frameworks and a Way Forward (2020) · cites this paper as Holland, 1986
In the citing paper
“When extant research finds the relationships between motivation and achievement, the interpretation with regards to causal relations remains difficult due to the lack of experimental manipulation (Granger, 1980;Holland, 1986;Marsh et al., 2018;Mega et al., 2014).”
From Motivation-Achievement Cycles in Learning: a Literature Review and Research Agenda (2022) · cites this paper as Holland, 1986
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Investigating causal relations by econometric models and cross-spectral methods
Granger, C. W. J. (1969) · Econometrica · doi:10.2307/1912791 · Cited by 22,989
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,804
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974
Rosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 6,472
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289
An Essay concerning Human Understanding
Locke, J. (1690) · Oxford University Press eBooks · doi:10.1093/oseo/instance.00018020 · Cited by 5,214
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
Bayesian inference for causal effects: The role of randomization
Rubin, D. B. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344064 · Cited by 2,503
Smoking and Carcinoma of the Lung
Doll, R., & Hill, A. B. (1950) · BMJ · doi:10.1136/bmj.2.4682.739 · Cited by 1,999
An Inquiry concerning human understanding
Hume, D. (1955) · Cited by 1,774
Granger, C. W. J. (1980) · Journal of Economic Dynamics and Control · doi:10.1016/0165-1889(80)90069-x · Cited by 1,746
Design and Analysis of Experiments
Ryan, T. P. (2007) · Wiley eBooks · doi:10.1002/9780470187548.ch12 · Cited by 1,435
Randomization Analysis of Experimental Data: The Fisher Randomization Test Comment
Rubin, D. B. (1980) · Journal of the American Statistical Association · doi:10.2307/2287653 · Cited by 1,368
Encyclopedia of Statistical Sciences.
Aspinall, L., Kotz, S., & Johnson, N. L. (1983) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347959 · Cited by 1,278
Mill, J. S. (1843) · Cited by 1,247
Introduction to Structural Equation Models.
Cherlin, A. J., & Duncan, O. D. (1976) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2064059 · Cited by 1,242
A Probabilistic Theory of Causality.
Good, I. J., & Suppes, P. (1972) · Journal of the American Statistical Association · doi:10.2307/2284742 · Cited by 1,240
Rosenbaum, P., & Rubin, D. B. (1983) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1983.tb01242.x · Cited by 1,128
The Arrangement of Field Experiments
Fisher, R. A. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-4380-9_8 · Cited by 977
Smoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions
Cornfield, J., Haenszel, W. M., Hammond, E., Lilienfeld, A. M., Shimkin, M. B., & Wynder, E. L. (1959) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/22.1.173 · Cited by 850
The Consquences of Adjustment for a Concomitant Variable That Has Been Affected by the Treatment
Rosenbaum, P. R. (1984) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981697 · Cited by 849
Structural Equation Models in the Social Sciences.
King, M. A., Goldberger, A. S., & Duncan, O. D. (1974) · The Economic Journal · doi:10.2307/2230507 · Cited by 803
Lung Cancer and Other Causes of Death in Relation to Smoking
Doll, R., & Hill, A. B. (1956) · BMJ · doi:10.1136/bmj.2.5001.1071 · Cited by 801
Journal of the National Cancer Institute
(1958) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/20.1.227 · Cited by 779
The Design and Analysis of Experiments
Kempthorne, O. (1952) · Soil Science · doi:10.1097/00010694-195205000-00012 · Cited by 621
Study of the Aetiology of Carcinoma of the Lung
Doll, R., & Hill, A. B. (1952) · BMJ · doi:10.1136/bmj.2.4797.1271 · Cited by 579
Journal of Educational Measurement
(2014) · Journal of Educational Measurement · doi:10.1111/(issn)1745-3984 · Cited by 567
The Bias Due to Incomplete Matching
Rosenbaum, P. R., & Rubin, D. B. (1985) · Biometrics · doi:10.2307/2530647 · Cited by 532
Bunge, M. (2017) · Routledge eBooks · doi:10.4324/9781315081656 · Cited by 495
Statistical Problems in Agricultural Experimentation
Neyman, J., Iwaszkiewicz, K., & Kołodziejczyk, S. (1935) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.2307/2983637 · Cited by 492
Causal Models in the Social Sciences.
Henry, N. W., & Blalock, H. M. (1976) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063121 · Cited by 474
Assignment to Treatment Group on the Basis of a Covariate
Rubin, D. B. (1977) · Journal of Educational Statistics · doi:10.2307/1164933 · Cited by 466
Introduction to Structural Equation Models
(1975) · Elsevier eBooks · doi:10.1016/c2009-0-03148-1 · Cited by 446
The Bias Due to Incomplete Matching
Rosenbaum, P. R. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810725.020 · Cited by 422
Rosenbaum, P. R. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477060 · Cited by 279
Kempthorne, O. (1978) · Biometrics · doi:10.2307/2529584 · Cited by 214
On the methodology of investigations of etiologic factors in chronic diseases
Yerushalmy, J., & Palmer, C. E. (1959) · Journal of Chronic Diseases · doi:10.1016/0021-9681(59)90015-3 · Cited by 205
Planning and Analysis of Observational Studies.
Sykes, R. E., & Cochran, W. G. (1984) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2067916 · Cited by 194
Planning and Analysis of Observational Studies
Cochran, W. G. (1983) · Wiley series in probability and statistics · doi:10.1002/9780470316542 · Cited by 170
PRODAN, M. (1968) · Elsevier eBooks · doi:10.1016/b978-0-08-012441-4.50022-5 · Cited by 164
Causal Models in the Social Sciences.
Barnes, J. A., & Blalock, H. M. (1972) · The Economic Journal · doi:10.2307/2231327 · Cited by 160
Conditional Permutation Tests and the Propensity Score in Observational Studies
Rosenbaum, P. R. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478082 · Cited by 156
CAUSATION AND DISEASE: A CHRONOLOGICAL JOURNEY
Evans, A. S. (1978) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112617 · Cited by 155
Principals of Modern Psychological Measurement
(2012) · Routledge eBooks · doi:10.4324/9780203056653 · Cited by 128
Holland, P. W., & Rubin, D. B. (1982) · ETS Research Report Series · doi:10.1002/j.2333-8504.1982.tb01321.x · Cited by 97
Self-citationRandomization Analysis of Experimental Data: The Fisher Randomization Test Rejoinder
Basu, D. (1980) · Journal of the American Statistical Association · doi:10.2307/2287654 · Cited by 82
Effects of self-study for coachable test item types.
Powers, D. E., & Swinton, S. S. (1984) · Journal of Educational Psychology · doi:10.1037/0022-0663.76.2.266 · Cited by 71
CHOOSING THE PARAMETER FOR A 2 × 2 TABLE OR A 2 × 2 × 2 TABLE ANALYSIS
Hamilton, M. A. (1979) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112689 · Cited by 65
A Linear Theory for Noncausality
Florens, J., & Mouchart, M. (1985) · Econometrica · doi:10.2307/1911729 · Cited by 60
Comment: Estimating the Effects Caused by Treatments
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477057 · Cited by 32
Conditional Permutation Tests and the Propensity Score in Observational Studies
Rosenbaum, P. R. (1984) · Journal of the American Statistical Association · doi:10.2307/2288402 · Cited by 24
Fisher, R. A. (1957) · BMJ · doi:10.1136/bmj.2.5035.43 · Cited by 15
(1870) · BMJ · doi:10.1136/bmj.1.478.209-a · Cited by 12
Government Says Cancer Rate Is Increasing
Smith, R. J. (1980) · Science · doi:10.1126/science.7403869 · Cited by 8
Causal Inference in Prospective and Retrospective Studies.
Holland, P. W., & Rubin, D. B. (1980) · Cited by 7
Self-citationCook, R. D. (1980) · Lecture notes in statistics · doi:10.1007/978-1-4612-6079-0_18 · Cited by 7
The Midwives Registration Bill Amended
Wall, W. B. (1890) · BMJ · doi:10.1136/bmj.2.1561.1272-b · Cited by 4
Shaw, R. E. (1952) · BMJ · doi:10.1136/bmj.2.4774.43-e · Cited by 2
Research works printed but not linked: 5
- 10.1016/B978-0-12-224150-5.50005-8 · DOI
- Hume , D. 1740 . A Treatise on Human Nature
- 10.1080/00031305.1985.10479383 · DOI
- 10.1080/01621459.1980.10477517 · DOI · Paper page
- Saris , W. and Stronkhorst , H. 1984 . Causal Modelling in Non-experimental Research , Amsterdam : Sociometric Research Foundation .
Cited by
Sorted by citations
Mediation in experimental and nonexperimental studies: New procedures and recommendations.
Shrout, P. E., & Bolger, N. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.4.422 · Cited by 11,335
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Difference-in-differences with variation in treatment timing
Goodman-Bacon, A. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2021.03.014 · Cited by 8,046
Metadata
- Authors
- Holland, P. W.
- Affiliations
- Educational Testing Service (US)
- Corresponding author
- Holland, Paul W.
- ORCID
- Holland, Paul W.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 1986-12
- Volume
- 81
- Issue
- 396
- Pages
- 945-960
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W1978108654
- Cited by
- 5,293
- References
- 62
- Citations by year
- 2026: 188 · 2025: 260 · 2024: 246 · 2023: 266 · 2022: 274 · 2021: 323 · 2020: 302 · 2019: 269 · 2018: 259 · 2017: 202 · 2016: 212 · 2015: 229 · 2014: 271 · 2013: 224 · 2012: 187
- FWCI
- 14.28
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- causal inference, causal models, correlation versus causation, randomized experiments, counterfactual reasoning, philosophy of statistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
