Statistics and Causal Inference
Abstract
Problems involving causal inference have dogged at the heels of statistics since its earliest days. Correlation does not imply causation, and yet causal conclusions drawn from a carefully designed experiment are often valid. What can a statistical model say about causation? This question is addressed by using a particular model for causal inference (Holland and Rubin 1983; Rubin 1974) to critique the discussions of other writers on causation and causal inference. These include selected philosophers, medical researchers, statisticians, econometricians, and proponents of causal modeling.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Due to the fundamental problem of causal inference, only one of the two potential outcomes is observed for each unit.
“As is usually the case, due to the fundamental problem of causal inference (Holland, 1986), we only observe one of the two potential outcomes for each unit i .”
From What's trending in difference-in-differences? A synthesis of the recent econometrics literature (2023) · cites this paper as Holland, 1986
Causal effects cannot be observed directly.
“causal effects can never be observed directly”
From Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable (2014) · cites this paper as Holland 1986
In uplift modeling, the actual outcome that is estimated is unobserved, so the error made by the model cannot be measured at the individual customer level.
“In uplift modeling, however, the actual outcome that is estimated is unobserved, and therefore the error made by the model cannot be measured at the individual customer level.”
From Why you should stop predicting customer churn and start using uplift models (2021) · cites this paper as [34]
Historical data containing information on observed customer behavior never allows classifying customers in the four groups shown in Table 1, as the net effect of treatment cannot be observed for an individual customer due to the fundamental problem of causal inference.
“Historical data containing information on observed customer behavior never allows classifying customers in the four groups shown in Table 1, as the net e ↵ ect of treatment cannot be observed for an individual customer due to the fundamental problem of causal inference [34].”
From Why you should stop predicting customer churn and start using uplift models (2021) · cites this paper as [34]
Uplift, or whether a customer is persuadable, a lost cause, do-not-disturb or a sure thing, cannot be observed due to the fundamental problem of causal inference.
“Uplift or, in other words, whether a customer is persuadable, a lost cause, do-notdisturb or a sure thing, cannot be observed due to the fundamental problem of causal inference [34].”
From Why you should stop predicting customer churn and start using uplift models (2021) · cites this paper as [34]
Because the antecedent variable is experimental in a randomized study, its causal relation to mediator and outcome can be interpreted unambiguously.
“Since the antecedent variable is an experimental variable in a randomized study, its causal relation to the mediator and the outcome variable can be interpreted unambiguously102,103.”
From Near transfer to an unrelated N-back task mediates the effect of N-back working memory training on matrix reasoning (2022) · cites this paper as [102]
Key to the ability to make inferences about mean causal effects are the assumptions of unit homogeneity and independence of units.
“Key to the ability to make inferences about mean causal effects are the assumptions of unit homogeneity and independence of units (Holland, 1986; King, Keohane and Verba, 1994: 91-97).”
From Multi-Method Research in the Social Sciences: A Review of Recent Frameworks and a Way Forward (2020) · cites this paper as Holland, 1986
When extant research finds relationships between motivation and achievement, the interpretation with regards to causal relations remains difficult due to the lack of experimental manipulation.
“When extant research finds the relationships between motivation and achievement, the interpretation with regards to causal relations remains difficult due to the lack of experimental manipulation (Granger, 1980;Holland, 1986;Marsh et al., 2018;Mega et al., 2014).”
From Motivation-Achievement Cycles in Learning: a Literature Review and Research Agenda (2022) · cites this paper as Holland, 1986
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241 · see citing works
Investigating causal relations by econometric models and cross-spectral methods
Granger, C. W. J. (1969) · Econometrica · doi:10.2307/1912791 · Cited by 22,989 · see citing works
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436 · see citing works
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,302 · see citing works
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · see citing works
Rosenbaum, P., & Rubin, D. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 6,854 · see citing works
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289 · see citing works
An Essay concerning Human Understanding
Locke, J. (1690) · Oxford University Press eBooks · doi:10.1093/oseo/instance.00018020 · Cited by 5,214 · see citing works
Rosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.1080/00031305.1985.10479383 · Cited by 4,415 · see citing works
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082 · see citing works
Bayesian inference for causal effects: The role of randomization
Rubin, D. B. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344064 · Cited by 2,503 · see citing works
Smoking and Carcinoma of the Lung
Doll, R., & Hill, A. B. (1950) · BMJ · doi:10.1136/bmj.2.4682.739 · Cited by 1,999 · see citing works
An Inquiry concerning human understanding
Hume, D. (1955) · Cited by 1,774 · see citing works
Granger, C. W. J. (1980) · Journal of Economic Dynamics and Control · doi:10.1016/0165-1889(80)90069-x · Cited by 1,746 · see citing works
Design and Analysis of Experiments
Ryan, T. P. (2007) · doi:10.1002/9780470187548.ch12 · Cited by 1,436 · see citing works
Randomization Analysis of Experimental Data: The Fisher Randomization Test Comment
Rubin, D. B. (1980) · Journal of the American Statistical Association · doi:10.2307/2287653 · Cited by 1,368 · see citing works
Encyclopedia of Statistical Sciences.
Aspinall, L., Kotz, S., & Johnson, N. L. (1983) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347959 · Cited by 1,271 · see citing works
Rosenbaum, P., & Rubin, D. (1983) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1983.tb01242.x · Cited by 1,264 · see citing works
Introduction to Structural Equation Models.
Cherlin, A. J., & Duncan, O. D. (1976) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2064059 · Cited by 1,242 · see citing works
A Probabilistic Theory of Causality.
Good, I. J., & Suppes, P. (1972) · Journal of the American Statistical Association · doi:10.2307/2284742 · Cited by 1,240 · see citing works
Mill, J. S. (1843) · Cited by 1,235 · see citing works
The Arrangement of Field Experiments
Fisher, R. A. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-4380-9_8 · Cited by 979 · see citing works
The Consquences of Adjustment for a Concomitant Variable That Has Been Affected by the Treatment
Rosenbaum, P. R. (1984) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981697 · Cited by 849 · see citing works
Smoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions
Cornfield, J., Haenszel, W. M., Hammond, E., Lilienfeld, A. M., Shimkin, M. B., & Wynder, E. L. (1959) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/22.1.173 · Cited by 842 · see citing works
Structural Equation Models in the Social Sciences.
King, M. A., Goldberger, A. S., & Duncan, O. D. (1974) · The Economic Journal · doi:10.2307/2230507 · Cited by 803 · see citing works
Lung Cancer and Other Causes of Death in Relation to Smoking
Doll, R., & Hill, A. B. (1956) · BMJ · doi:10.1136/bmj.2.5001.1071 · Cited by 801 · see citing works
Journal of the National Cancer Institute
(1958) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/20.1.227 · Cited by 779 · see citing works
Journal of Educational Measurement
(2014) · Journal of Educational Measurement · doi:10.1111/(issn)1745-3984 · Cited by 567 · see citing works
Study of the Aetiology of Carcinoma of the Lung
Doll, R., & Hill, A. B. (1952) · BMJ · doi:10.1136/bmj.2.4797.1271 · Cited by 563 · see citing works
The Bias Due to Incomplete Matching
Rosenbaum, P. R., & Rubin, D. B. (1985) · Biometrics · doi:10.2307/2530647 · Cited by 524 · see citing works
The Design and Analysis of Experiments
Kempthorne, O. (1952) · Soil Science · doi:10.1097/00010694-195205000-00012 · Cited by 522 · see citing works
Bunge, M. (2017) · Routledge eBooks · doi:10.4324/9781315081656 · Cited by 495 · see citing works
Statistical Problems in Agricultural Experimentation
Neyman, J., Iwaszkiewicz, K., & Kołodziejczyk, S. (1935) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.2307/2983637 · Cited by 492 · see citing works
Causal Models in the Social Sciences.
Henry, N. W., & Blalock, H. M. (1976) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2063121 · Cited by 474 · see citing works
Assignment to Treatment Group on the Basis of a Covariate
Rubin, D. B. (1977) · Journal of Educational Statistics · doi:10.2307/1164933 · Cited by 466 · see citing works
Introduction to Structural Equation Models
(1975) · Elsevier eBooks · doi:10.1016/c2009-0-03148-1 · Cited by 446 · see citing works
The Bias Due to Incomplete Matching
Rosenbaum, P. R. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810725.020 · Cited by 422 · see citing works
Rosenbaum, P. R. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477060 · Cited by 279 · see citing works
Kempthorne, O. (1978) · Biometrics · doi:10.2307/2529584 · Cited by 214 · see citing works
On the methodology of investigations of etiologic factors in chronic diseases
Yerushalmy, J., & Palmer, C. E. (1959) · Journal of Chronic Diseases · doi:10.1016/0021-9681(59)90015-3 · Cited by 205 · see citing works
Planning and Analysis of Observational Studies.
Sykes, R. E., & Cochran, W. G. (1984) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2067916 · Cited by 194 · see citing works
Planning and Analysis of Observational Studies
Cochran, W. G. (1983) · Wiley series in probability and statistics · doi:10.1002/9780470316542 · Cited by 168 · see citing works
PRODAN, M. (1968) · Elsevier eBooks · doi:10.1016/b978-0-08-012441-4.50022-5 · Cited by 164 · see citing works
Causal Models in the Social Sciences.
Barnes, J. A., & Blalock, H. M. (1972) · The Economic Journal · doi:10.2307/2231327 · Cited by 160 · see citing works
Conditional Permutation Tests and the Propensity Score in Observational Studies
Rosenbaum, P. R. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478082 · Cited by 157 · see citing works
CAUSATION AND DISEASE: A CHRONOLOGICAL JOURNEY
Evans, A. S. (1978) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112617 · Cited by 155 · see citing works
Principals of Modern Psychological Measurement
(2012) · Routledge eBooks · doi:10.4324/9780203056653 · Cited by 128 · see citing works
Holland, P. W., & Rubin, D. B. (1982) · ETS Research Report Series · doi:10.1002/j.2333-8504.1982.tb01321.x · Cited by 97 · see citing works
Self-citationRubin, D. B. (1980) · Journal of the American Statistical Association · doi:10.1080/01621459.1980.10477517 · Cited by 83 · see citing works
Randomization Analysis of Experimental Data: The Fisher Randomization Test Rejoinder
Basu, D. (1980) · Journal of the American Statistical Association · doi:10.2307/2287654 · Cited by 81 · see citing works
Effects of self-study for coachable test item types.
Powers, D. E., & Swinton, S. S. (1984) · Journal of Educational Psychology · doi:10.1037/0022-0663.76.2.266 · Cited by 69 · see citing works
CHOOSING THE PARAMETER FOR A 2 × 2 TABLE OR A 2 × 2 × 2 TABLE ANALYSIS
Hamilton, M. A. (1979) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112689 · Cited by 65 · see citing works
A Linear Theory for Noncausality
Florens, J., & Mouchart, M. (1985) · Econometrica · doi:10.2307/1911729 · Cited by 60 · see citing works
Comment: Estimating the Effects Caused by Treatments
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477057 · Cited by 32 · see citing works
Conditional Permutation Tests and the Propensity Score in Observational Studies
Rosenbaum, P. R. (1984) · Journal of the American Statistical Association · doi:10.2307/2288402 · Cited by 24 · see citing works
Fisher, R. A. (1957) · BMJ · doi:10.1136/bmj.2.5035.43 · Cited by 15 · see citing works
(1870) · BMJ · doi:10.1136/bmj.1.478.209-a · Cited by 12 · see citing works
Government Says Cancer Rate Is Increasing
Smith, R. J. (1980) · Science · doi:10.1126/science.7403869 · Cited by 8 · see citing works
Causal Inference in Prospective and Retrospective Studies.
Holland, P. W., & Rubin, D. B. (1980) · Cited by 7 · see citing works
Self-citationCook, R. D. (1980) · Lecture notes in statistics · doi:10.1007/978-1-4612-6079-0_18 · Cited by 7 · see citing works
The Midwives Registration Bill Amended
Wall, W. B. (1890) · BMJ · doi:10.1136/bmj.2.1561.1272-b · Cited by 4 · see citing works
Shaw, R. E. (1952) · BMJ · doi:10.1136/bmj.2.4774.43-e · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Mediation in experimental and nonexperimental studies: New procedures and recommendations.
Shrout, P. E., & Bolger, N. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.4.422 · Cited by 11,335 · see citing works
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467 · see citing works
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · see citing works
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613 · see citing works
Difference-in-differences with variation in treatment timing
Goodman-Bacon, A. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2021.03.014 · Cited by 8,046 · see citing works
SOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
Caliendo, M., & Kopeinig, S. (2008) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2007.00527.x · Cited by 6,238 · see citing works
Matching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478 · see citing works
Recent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.1.5 · Cited by 4,926 · see citing works
Propensity Score-Matching Methods for Nonexperimental Causal Studies
Dehejia, R., & Wahba, S. (2002) · The Review of Economics and Statistics · doi:10.1162/003465302317331982 · Cited by 4,777 · see citing works
Edwards, J. R., & Lambert, L. S. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.1.1 · Cited by 4,711 · see citing works
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230 · see citing works
Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference
Ho, D. E., Imai, K., King, G., & Stuart, E. (2007) · Political Analysis · doi:10.1093/pan/mpl013 · Cited by 4,411 · see citing works
A general approach to causal mediation analysis.
Imai, K., Keele, L., & Tingley, D. (2010) · Psychological Methods · doi:10.1037/a0020761 · Cited by 4,143 · see citing works
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · see citing works
Regression discontinuity designs: A guide to practice
Imbens, G. W., & Lemieux, T. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.001 · Cited by 3,701 · see citing works
Cole, D. A., & Maxwell, S. E. (2003) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.112.4.558 · Cited by 3,605 · see citing works
Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review
Imbens, G. W. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023651 · Cited by 2,994 · see citing works
The Economics and Econometrics of Active Labor Market Programs
Heckman, J. J., Lalonde, R. J., & Smith, J. A. (1999) · Handbook of labour economics · doi:10.1016/s1573-4463(99)03012-6 · Cited by 2,992 · see citing works
Some recent development in a concept of causality
Granger, C. W. J. (1988) · Journal of Econometrics · doi:10.1016/0304-4076(88)90045-0 · Cited by 2,982 · see citing works
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791 · see citing works
Endogeneity in Empirical Corporate Finance1
Roberts, M. R., & Whited, T. M. (2013) · Handbook of the Economics of Finance · doi:10.1016/b978-0-44-453594-8.00007-0 · Cited by 2,459 · see citing works
Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395 · see citing works
Angrist, J. D., & Krueger, A. B. (2001) · Journal of Economic Perspectives · doi:10.1257/jep.15.4.69 · Cited by 2,355 · see citing works
Causal Inference Using Potential Outcomes: Design, Modeling, Decisions
Rubin, D. B. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000001880 · Cited by 2,173 · see citing works
THE EFFECTS OF STRESSFUL LIFE EVENTS ON DEPRESSION
Kessler, R. C. (1997) · Annual Review of Psychology · doi:10.1146/annurev.psych.48.1.191 · Cited by 2,111 · see citing works
What's trending in difference-in-differences? A synthesis of the recent econometrics literature
Roth, J., Sant’Anna, P. H. C., Bilinski, A., & Poe, J. (2023) · Journal of Econometrics · doi:10.1016/j.jeconom.2023.03.008 · Cited by 2,027 · see citing works
“As is usually the case, due to the fundamental problem of causal inference (Holland, 1986), we only observe one of the two potential outcomes for each unit i .”
(2009)
Hainmueller, J., Hopkins, D. J., & Yamamoto, T. (2014) · Political Analysis · doi:10.1093/pan/mpt024 · Cited by 1,990 · see citing works
Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA
Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K., & Larson, H. J. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01056-1 · Cited by 1,800 · see citing works
Quantifying the influence of climate on human conflict
Hsiang, S. M., Burke, M., & Miguel, E. (2013) · Science · doi:10.1126/science.1235367 · Cited by 1,827 · see citing works
Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001) · American Journal of Psychiatry · doi:10.1176/appi.ajp.158.6.848 · Cited by 1,712 · see citing works
Tunis (2003) · doi:10.1001/jama.290.12.1624 · Cited by 1,724 · see citing works
Why Propensity Scores Should Not Be Used for Matching
King, G., & Nielsen, R. (2019) · Political Analysis · doi:10.1017/pan.2019.11 · Cited by 1,706 · see citing works
Empirical Strategies in Labor Economics
Angrist, J. D., & Krueger, A. B. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03004-7 · Cited by 1,628 · see citing works
Recursive partitioning for heterogeneous causal effects
Athey, S., & Imbens, G. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510489113 · Cited by 1,586 · see citing works
The theoretical status of latent variables.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003) · Psychological Review · doi:10.1037/0033-295x.110.2.203 · Cited by 1,566 · see citing works
Sekhon, J. S. (2011) · Journal of Statistical Software · doi:10.18637/jss.v042.i07 · Cited by 1,521 · see citing works
IMAI, K., KEELE, L., TINGLEY, D., & YAMAMOTO, T. (2011) · American Political Science Review · doi:10.1017/s0003055411000414 · Cited by 1,537 · see citing works
Principal Stratification in Causal Inference
Frangakis, C., & Rubin, D. B. (2002) · Biometrics · doi:10.1111/j.0006-341x.2002.00021.x · Cited by 1,496 · see citing works
Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects
Abadie, A. (2021) · Journal of Economic Literature · doi:10.1257/jel.20191450 · Cited by 1,486 · see citing works
A tutorial on propensity score estimation for multiple treatments using generalized boosted models
McCaffrey, D. F., Griffin, B. A., Almirall, D., Slaughter, M. E., Ramchand, R., & Burgette, L. F. (2013) · Statistics in Medicine · doi:10.1002/sim.5753 · Cited by 1,450 · see citing works
Reducing bias through directed acyclic graphs
Shrier, I., & Platt, R. W. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-70 · Cited by 1,394 · see citing works
Iacobucci, D., Saldanha, N., & Deng, X. (2007) · Journal of Consumer Psychology · doi:10.1016/s1057-7408(07)70020-7 · Cited by 1,362 · see citing works
Social and economic impacts of climate
Carleton, T. A., & Hsiang, S. M. (2016) · Science · doi:10.1126/science.aad9837 · Cited by 1,348 · see citing works
Angrist, J. D., & Imbens, G. W. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476535 · Cited by 1,346 · see citing works
Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005) · Prevention Science · doi:10.1007/s11121-005-5553-y · Cited by 1,338 · see citing works
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.403 · Cited by 1,317 · see citing works
Statistical Thinking in Empirical Enquiry
Wild, C. J., & Pfannkuch, M. (1999) · International Statistical Review · doi:10.1111/j.1751-5823.1999.tb00442.x · Cited by 1,278 · see citing works
Cross-Sectional versus longitudinal survey research: concepts, findings, and guidelines
Rindfleisch, A., Malter, A. J., Ganesan, S., & Moorman, C. (2008) · Journal of Marketing Research · doi:10.1509/jmkr.45.3.261 · Cited by 1,305 · see citing works
Kang, J. D. Y., & Schafer, J. L. (2007) · Statistical Science · doi:10.1214/07-sts227 · Cited by 1,259 · see citing works
Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable
Elwert, F., & Winship, C. (2014) · Annual Review of Sociology · doi:10.1146/annurev-soc-071913-043455 · Cited by 1,168 · see citing works
Zapf, D., Dormann, C., & Fresé, M. (1996) · Journal of Occupational Health Psychology · doi:10.1037//1076-8998.1.2.145 · Cited by 1,174 · see citing works
Decomposition Methods in Economics
Fortin, N. M., Lemieux, T., & FIRPO, S. P. (2011) · Handbook of labour economics · doi:10.1016/s0169-7218(11)00407-2 · Cited by 1,162 · see citing works
Machine Learning Methods That Economists Should Know About
Athey, S., & Imbens, G. W. (2019) · Annual Review of Economics · doi:10.1146/annurev-economics-080217-053433 · Cited by 1,163 · see citing works
Diamond, A., & Sekhon, J. S. (2012) · The Review of Economics and Statistics · doi:10.1162/rest_a_00318 · Cited by 1,066 · see citing works
Correlation and Simple Linear Regression
Zou, K. H., Tuncali, K., & Silverman, S. G. (2003) · Radiology · doi:10.1148/radiol.2273011499 · Cited by 1,096 · see citing works
Rubin, D. B. (2007) · Statistics in Medicine · doi:10.1002/sim.2739 · Cited by 1,047 · see citing works
Impact of a Confounding Variable on a Regression Coefficient
Frank, K. A. (2000) · Sociological Methods & Research · doi:10.1177/0049124100029002001 · Cited by 1,042 · see citing works
Resources, Instruction, and Research
Cohen, D., Raudenbush, S., & Ball, D. (2003) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737025002119 · Cited by 1,061 · see citing works
The economics and psychology of personality traits
Borghans, L., Duckworth, A. L., Heckman, J. J., & ter Weel, B. (2008) · Research Publications (Maastricht University) · doi:10.26481/umaror.2008001 · Cited by 974 · see citing works
Panel Models in Sociological Research: Theory into Practice
Halaby, C. N. (2004) · Annual Review of Sociology · doi:10.1146/annurev.soc.30.012703.110629 · Cited by 946 · see citing works
Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models
Xu, Y. (2017) · Political Analysis · doi:10.1017/pan.2016.2 · Cited by 938 · see citing works
Confounding and Collapsibility in Causal Inference
Greenland, S., Pearl, J., & Robins, J. M. (1999) · Statistical Science · doi:10.1214/ss/1009211805 · Cited by 933 · see citing works
A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research
Mahoney, J., & Goertz, G. (2006) · Political Analysis · doi:10.1093/pan/mpj017 · Cited by 904 · see citing works
THE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA
Winship, C., & Morgan, S. L. (1999) · Annual Review of Sociology · doi:10.1146/annurev.soc.25.1.659 · Cited by 890 · see citing works
Parental involvement on student academic achievement: A meta-analysis
Castro, M., Expósito-Casas, E., López-Martín, E., Lizasoain, L., Navarro-Asencio, E., & Gaviria, J. L. (2015) · Educational Research Review · doi:10.1016/j.edurev.2015.01.002 · Cited by 983 · see citing works
Chief Executive Officer Equity Incentives and Accounting Irregularities
Armstrong, C., Jagolinzer, A. D., & Larcker, D. F. (2009) · Journal of Accounting Research · doi:10.1111/j.1475-679x.2009.00361.x · Cited by 882 · see citing works
Causal Inference With General Treatment Regimes
Imai, K., & van Dyk, D. V. (2004) · Journal of the American Statistical Association · doi:10.1198/016214504000001187 · Cited by 1,022 · see citing works
Health metrics and evaluation: strengthening the science
Murray, C. J., & Frenk, J. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60526-7 · Cited by 849 · see citing works
Estimating and testing mediation and moderation in within-subject designs.
Judd, C. M., Kenny, D. A., & McClelland, G. H. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.2.115 · Cited by 846 · see citing works
For objective causal inference, design trumps analysis
Rubin, D. B. (2008) · The Annals of Applied Statistics · doi:10.1214/08-aoas187 · Cited by 831 · see citing works
Behavioral science at the crossroads in public health: Extending horizons, envisioning the future
Glass, T. A., & McAtee, M. J. (2006) · Social Science & Medicine · doi:10.1016/j.socscimed.2005.08.044 · Cited by 821 · see citing works
Dissecting the Social: On the Principles of Analytical Sociology
Hedstrom, P. (2005) · doi:10.1017/cbo9780511488801 · Cited by 816 · see citing works
Models for Sample Selection Bias
Winship, C., & Mare, R. D. (1992) · Annual Review of Sociology · doi:10.1146/annurev.so.18.080192.001551 · Cited by 812 · see citing works
Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs
Dehejia, R., & Wahba, S. (1998) · National Bureau of Economic Research · doi:10.3386/w6586 · Cited by 821 · see citing works
Advances in Mediation Analysis: A Survey and Synthesis of New Developments
Preacher, K. J. (2014) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015258 · Cited by 800 · see citing works
Some Recent Developments in a Concept of Causality
Granger, C. W. J. (2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511753978.004 · Cited by 790 · see citing works
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · Thammasat University Digital Collections · Cited by 838 · see citing works
(2005) · Cited by 772 · see citing works
Civil conflicts are associated with the global climate
Hsiang, S., Meng, K. C., & Cane, M. A. (2011) · Nature · doi:10.1038/nature10311 · Cited by 776 · see citing works
Little, R. J., & Rubin, D. B. (2000) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.21.1.121 · Cited by 721 · see citing works
Decomposition Methods in Economics
Fortin, N. M., Lemieux, T., & FIRPO, S. P. (2010) · National Bureau of Economic Research · doi:10.3386/w16045 · Cited by 746 · see citing works
Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement<sup>*</sup>
Rothstein, J. (2010) · Quarterly Journal of Economics · doi:10.1162/qjec.2010.125.1.175 · Cited by 735 · see citing works
DiPrete, T. A., & Gangl, M. (2004) · Sociological Methodology · doi:10.1111/j.0081-1750.2004.00154.x · Cited by 734 · see citing works
Lechner, M. (2001) · ZEW economic studies · doi:10.1007/978-3-642-57615-7_3 · Cited by 728 · see citing works
Healthy, wealthy, and wise? Tests for direct causal paths between health and socioeconomic status
Adams, P., Hurd, M. D., McFadden, D., Merrill, A., & Ribeiro, T. (2003) · Journal of Econometrics · doi:10.1016/s0304-4076(02)00145-8 · Cited by 709 · see citing works
Domestic Audience Costs in International Relations: An Experimental Approach
Tomz, M. R. (2007) · International Organization · doi:10.1017/s0020818307070282 · Cited by 699 · see citing works
Matched Sampling for Causal Effects: Contributor Acknowledgments
Rubin, D. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810725 · Cited by 287 · see citing works
Lauritzen, S., & Wermuth, N. (1989) · The Annals of Statistics · doi:10.1214/aos/1176347003 · Cited by 841 · see citing works
Eight Myths About Causality and Structural Equation Models
Bollen, K. A., & Pearl, J. (2013) · Handbooks of sociology and social research · doi:10.1007/978-94-007-6094-3_15 · Cited by 680 · see citing works
Rubin, D. B. (1990) · Statistical Science · doi:10.1214/ss/1177012032 · Cited by 665 · see citing works
Burke, M., Hsiang, S. M., & Miguel, E. (2015) · Annual Review of Economics · doi:10.1146/annurev-economics-080614-115430 · Cited by 655 · see citing works
Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique
Lin, W. T. (2013) · The Annals of Applied Statistics · doi:10.1214/12-aoas583 · Cited by 633 · see citing works
Jusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romić, I., Wang, Z., Geček, S., Lipić, T., Boris Podobnik, A., Wang, L., et al. (2022) · Physics Reports · doi:10.1016/j.physrep.2021.10.005 · Cited by 662 · see citing works
Harding, D. J. (2003) · American Journal of Sociology · doi:10.1086/379217 · Cited by 640 · see citing works
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633 · see citing works
Efficient semiparametric estimation of multi-valued treatment effects under ignorability
Cattaneo, M. D. (2009) · Journal of Econometrics · doi:10.1016/j.jeconom.2009.09.023 · Cited by 621 · see citing works
Tarka, P. (2017) · Quality & Quantity · doi:10.1007/s11135-017-0469-8 · Cited by 645 · see citing works
The Nonparametric Identification of Treatment Effects in Duration Models
Abbring, J. H., & van den Berg, G. J. (2003) · Econometrica · doi:10.1111/1468-0262.00456 · Cited by 616 · see citing works
Ariel, B., Farrar, W. A., & Sutherland, A. (2015) · Journal of Quantitative Criminology · doi:10.1007/s10940-014-9236-3 · Cited by 621 · see citing works
- Authors
- Holland, P. W.
- Affiliations
- Educational Testing Service (US)
- Corresponding author
- Holland, Paul W.
- ORCID
- Holland, Paul W.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 1986-12
- Volume
- 81
- Issue
- 396
- Pages
- 945-960
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W1978108654
- Cited by
- 5,293
- References
- 62
- Citations by year
- 2026: 188 · 2025: 260 · 2024: 246 · 2023: 266 · 2022: 274 · 2021: 323 · 2020: 302 · 2019: 269 · 2018: 259 · 2017: 202 · 2016: 212 · 2015: 229 · 2014: 271 · 2013: 224 · 2012: 187
- FWCI
- 14.28
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- causal inference, causal models, correlation versus causation, randomized experiments, counterfactual reasoning, philosophy of statistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review