The DOI matches its Crossref record; 46 of the 47 references it cites were found in a registry or an index.
Thinking Clearly About Correlations and Causation: Graphical Causal Models for Observational Data
Open accessAbstract
Correlation does not imply causation; but often, observational data are the only option, even though the research question at hand involves causality. This article discusses causal inference based on observational data, introducing readers to graphical causal models that can provide a powerful tool for thinking more clearly about the interrelations between variables. Topics covered include the rationale behind the statistical control of third variables, common procedures for statistical control, and what can go wrong during their implementation. Certain types of third variables—colliders and mediators—should not be controlled for because that can actually move the estimate of an association away from the value of the causal effect of interest. More subtle variations of such harmful control include using unrepresentative samples, which can undermine the validity of causal conclusions, and statistically controlling for mediators. Drawing valid causal inferences on the basis of observational data is not a mechanistic procedure but rather always depends on assumptions that require domain knowledge and that can be more or less plausible. However, this caveat holds not only for research based on observational data, but for all empirical research endeavors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532 · cited 1×
“Both old and newer common approaches to estimating the remaining (direct) effect after accounting for a mediator (e.g., Baron & Kenny, 1986; Hayes, 2009) rely on statistical control of the mediating variable, but such approaches can introduce endogenous selection bias (Elwert & Winship, 2014).”
BackgroundMissing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638 · cited 1×
“Schafer and Graham (2002) have provided an introduction to the management of missing data for psychologists.”
Background(1979) · Medical Entomology and Zoology · Cited by 11,252
Beyond Baron and Kenny: Statistical Mediation Analysis in the New Millennium
Hayes, A. F. (2009) · Communication Monographs · doi:10.1080/03637750903310360 · Cited by 9,757 · cited 1×
“Both old and newer common approaches to estimating the remaining (direct) effect after accounting for a mediator (e.g., Baron & Kenny, 1986; Hayes, 2009) rely on statistical control of the mediating variable, but such approaches can introduce endogenous selection bias (Elwert & Winship, 2014).”
BackgroundEstimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436 · cited 1×
“A solid rule of thumb is that researchers should not control for such posttreatment variables (Rosenbaum, 1984; Rubin, 1974).”
BackgroundCausation, Prediction, and Search
Spirtes, P., Glymour, C., & Scheines, R. (2001) · doi:10.7551/mitpress/1754.001.0001 · Cited by 4,521
Causal diagrams for empirical research
PEARL, J. (1995) · Biometrika · doi:10.1093/biomet/82.4.669 · Cited by 2,355 · cited 1×
“They were developed primarily by the computer scientist Judea Pearl (e.g., Pearl, 1995; see Pearl, Glymour, & Jewell, 2016, for an introduction) and share many features with structural equation models (SEMs).1 DAGs offer an intuitive approach for thinking about causal structures.”
MethodWhy Propensity Scores Should Not Be Used for Matching
King, G., & Nielsen, R. (2019) · Political Analysis · doi:10.1017/pan.2019.11 · Cited by 1,706 · cited 1×
“The use of propensity scores for matching is controversial, and critics have indicated that other procedures might be preferable (King & Nielsen, 2016).”
BackgroundThe theoretical status of latent variables.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003) · Psychological Review · doi:10.1037/0033-295x.110.2.203 · Cited by 1,566
Angrist, J. D., & Pischke, J.-S. (2010) · Journal of Economic Perspectives · doi:10.1257/jep.24.2.3 · Cited by 1,566
Increasing Transparency Through a Multiverse Analysis
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616658637 · Cited by 1,512 · cited 1×
“Jackson, Thoemmes, Jonkmann, Lüdtke, and Trautwein (2012) were interested in the effects of military training (in comparison with civilian community service) on personality.”
BackgroundHaving less, giving more: The influence of social class on prosocial behavior.
Piff, P. K., Kraus, M. W., Côté, S., Cheng, B. H., & Keltner, D. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0020092 · Cited by 1,478
Causal Inference in Statistics
Pearl, J., Glymour, M., & Jewell, N. P. (2016) · Cited by 1,436
Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable
Elwert, F., & Winship, C. (2014) · Annual Review of Sociology · doi:10.1146/annurev-soc-071913-043455 · Cited by 1,168 · cited 3×
“Elwert and Winship (2014) provided a succinct summary of the many ways in which endogenous selection bias can arise.”
BackgroundMethodSmith, G. C. S., & Pell, J. P. (2003) · BMJ · doi:10.1136/bmj.327.7429.1459 · Cited by 1,155
Methodological Urban Legends: The Misuse of Statistical Control Variables
Spector, P. E., & Brannick, M. T. (2011) · Organizational Research Methods · doi:10.1177/1094428110369842 · Cited by 1,146 · cited 1×
“…resulting from such an approach have been labeled “garbage-can regressions” (Achen, 2005) because the idea that the inclusion of a multitude of control variables will necessarily improve (and will not worsen) causal inference is a methodological urban legend at best (Spector & Brannick, 2011).”
BackgroundYes, but what’s the mechanism? (don’t expect an easy answer).
Bullock, J. G., Green, D. P., & Ha, S. E. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0018933 · Cited by 1,063
Natural Experiments in the Social Sciences
Dunning, T. (2012) · Cambridge University Press eBooks · doi:10.1017/cbo9781139084444 · Cited by 980
The Consquences of Adjustment for a Concomitant Variable That Has Been Affected by the Treatment
Rosenbaum, P. R. (1984) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981697 · Cited by 849 · cited 1×
“A solid rule of thumb is that researchers should not control for such posttreatment variables (Rosenbaum, 1984; Rubin, 1974).”
BackgroundQuantifying Biases in Causal Models: Classical Confounding vs Collider-Stratification Bias
Greenland, S. (2003) · Epidemiology · doi:10.1097/01.ede.0000042804.12056.6c · Cited by 810 · cited 1×
“In recent years, this potential source of bias has been pointed out in a variety of research fields, such as epidemiology (Greenland, 2003), personality psychology (Lee, 2012), and genetics (Munafò, Tilling, Taylor, Evans, & Smith, 2017).”
BackgroundThree Laws of Behavior Genetics and What They Mean
Turkheimer, E. (2000) · Current Directions in Psychological Science · doi:10.1111/1467-8721.00084 · Cited by 699 · cited 3×
“This example of genetic confounding is adapted from an example mentioned by Turkheimer (2000).”
BackgroundMethodInfluentialCartwright, N. D. (2007) · BioSocieties · doi:10.1017/s1745855207005029 · Cited by 588
Let's put garbage-can regressions and garbage-can probits where they belong
Achen, C. H. (2005) · Conflict Management and Peace Science · doi:10.1080/07388940500339167 · Cited by 576 · cited 2×
“Models resulting from such an approach have been labeled “garbage-can regressions” (Achen, 2005) because the idea that the inclusion of a multitude of control variables will necessarily improve (and will not worsen) causal inference is a methodological urban legend at best (Spector & Brannick,…”
BackgroundEuropean Journal of Personality
(2006) · European Journal of Personality · doi:10.1002/(issn)1099-0984 · Cited by 566
Statistically controlling for confounding constructs is harder than you think
Westfall, J., & Yarkoni, T. (2016) · PLOS ONE · doi:10.1371/journal.pone.0152719 · Cited by 547 · cited 3×
“Westfall and Yarkoni (2016) assessed how the false positive rate for an effect is affected by measurement error of covariates that are being controlled for.”
BackgroundInfluentialCounterfactuals and Causal Inference: Methods and Principles for Social Research
Hipp, J. R. (2008) · Contemporary Sociology A Journal of Reviews · doi:10.1177/009430610803700409 · Cited by 409
Military Training and Personality Trait Development
Jackson, J. J., Thoemmes, F. J., Jonkmann, K., Lüdtke, O., & Trautwein, U. (2012) · Psychological Science · doi:10.1177/0956797611423545 · Cited by 353
Social Psychology and Science: Some Lessons From Solomon Asch
Rozin, P. N. (2001) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0501_1 · Cited by 351 · cited 3×
“Schafer and Graham (2002) have provided an introduction to the management of missing data for psychologists.”
BackgroundMethodElwert, F. (2013) · Handbooks of Sociology and Social Research · doi:10.1007/978-94-007-6094-3_13 · Cited by 325 · cited 12×
“estimation of the identified effect are two distinct problems (Elwert, 2013): The right variables can be controlled for, but this can be done in the wrong way, as I discuss later.”
BackgroundInfluentialAn SEM approach to continuous time modeling of panel data: Relating authoritarianism and anomia.
Voelkle, M. C., Oud, J. H. L., Davidov, E., & Schmidt, P. (2012) · Psychological Methods · doi:10.1037/a0027543 · Cited by 320
[Bayesian Analysis in Expert Systems]: Comment: Graphical Models, Causality and Intervention
Pearl, J. (1993) · Statistical Science · doi:10.1214/ss/1177010894 · Cited by 258
The Emperor of All Maladies: A Biography of Cancer
Mehta, P. (2011) · JAMA · doi:10.1001/jama.2011.798 · Cited by 225
Reversing Arrows in Mediation Models Does Not Distinguish Plausible Models
Thoemmes, F. (2015) · Basic and Applied Social Psychology · doi:10.1080/01973533.2015.1049351 · Cited by 208
Replication and robustness in developmental research
Duncan, G. J., Engel, M., Claessens, A., & Dowsett, C. J. (2014) · Developmental Psychology · doi:10.1037/a0037996 · Cited by 207
The Signal and the Noise: The Art and Science of Prediction
Silver, N. (2012) · Medical Entomology and Zoology · Cited by 174
Sucar, L. E. (2020) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-61943-5_14 · Cited by 128
To Adjust or Not to Adjust? Sensitivity Analysis of M -Bias and Butterfly-Bias
Ding, P., & Miratrix, L. W. (2014) · Journal of Causal Inference · doi:10.1515/jci-2013-0021 · Cited by 126 · cited 1×
“Quite pragmatically, one can gauge which of the two biases is more problematic and then settle for the lesser evil (Ding & Miratrix, 2015).”
BackgroundCorrelation and Causation in the Study of Personality
Lee, J. J. (2012) · European Journal of Personality · doi:10.1002/per.1863 · Cited by 121 · cited 2×
“personality psychology (Lee, 2012), and genetics (Munafò, Tilling, Taylor, Evans, & Smith, 2017).”
BackgroundBehavior Genetic Research Methods: Testing Quasi-Causal Hypotheses Using Multivariate Twin Data
Turkheimer, E., & Harden, K. P. (2014) · Handbook of Research Methods in Social and Personality Psychology · doi:10.1017/cbo9780511996481.012 · Cited by 111
Pearl's D-Separation: One More Step Into Causal Thinking
Hayduk, L. A., Cummings, G. G., Stratkotter, R., Nimmo, M., Grygoryev, K., Dosman, D., Gillespie, M. W., Pazderka, H., & Boadu, K. (2003) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem1002_8 · Cited by 102 · cited 2×
“Accessible introductions to the underlying concept of d- separation, which determines whether paths transmit association or not, can be found in Hayduk et al. (2003) and, in a very brief form, in Thoemmes’s (2015) Appendix A. 6.”
BackgroundPropensity scores and M‐structures
Sjölander, A. (2009) · Statistics in Medicine · doi:10.1002/sim.3532 · Cited by 72 · cited 2×
“…do not change anything about the specifics of causal identification: If an important confounder is omitted, or if variables that should not be included are included (Sjölander, 2009), propensity scores fail to properly identify the causal effect, just as other methods of statistical adjustment do.”
BackgroundGraphical Representation of Missing Data Problems
Thoemmes, F. J., & Mohan, K. (2015) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.937378 · Cited by 68 · cited 2×
“For example, Turkheimer and Harden (2014) investigated whether religiosity has a causal effect on delinquency and found a negative correlation when they simply correlated the variables across their whole sample.”
BackgroundMethodCollider Scope: When selection bias can substantially influence observed associationsPreprint
Munafo, M. R., Tilling, K., Taylor, A. E., Evans, D. M., & Smith, G. D. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/079707 · Cited by 50
Tiefenbach, T., & Kohlbacher, F. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2015.05.040 · Cited by 39 · cited 1×
“…it has become common practice to include as many covariates as possible—to the point where authors imply or claim that they have additional confidence in their findings because, for example, their study “uses more control variables than previous studies” did (Tiefenbach & Kohlbacher, 2015, p. 85).”
ResultWhat Do the Items and Their Associations Refer to in a Network Approach to Personality?
Asendorpf, J. B. (2012) · European Journal of Personality · doi:10.1002/per.1867 · Cited by 14 · cited 1×
“Most substantive questions are concerned with causal effects, and, “as humans, we cannot avoid thinking in terms of causality” (Asendorpf, 2012, p. 391).”
BackgroundKievit, R., Waldorp, L., Kan, K., & Wicherts, J. M. (2012) · European Journal of Personality · doi:10.1002/per.1865 · Cited by 5
Research works printed but not linked: 2
- Mohan K., 2013, Advances in Neural Information Processing Systems 26 (NIPS 2013), 1277
- Robins J. M., 1999, Computation, Causation, & Discovery, 305
Cited by
Sorted by citations
Network analysis of multivariate data in psychological science
Borsboom, D., Deserno, M. K., Rhemtulla, M., Epskamp, S., Fried, E. I., McNally, R. J., Robinaugh, D. J., Perugini, M., Dalege, J., Costantini, G., et al. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-021-00055-w · Cited by 1,371
Scott, A. J., Webb, T. L., Martyn-St James, M., Rowse, G., & Weich, S. (2021) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2021.101556 · Cited by 1,094
Collider bias undermines our understanding of COVID-19 disease risk and severity
Griffith, G. J., Morris, T. T., Tudball, M. J., Herbert, A., Mancano, G., Pike, L., Sharp, G. C., Sterne, J., Palmer, T. M., Davey Smith, G., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19478-2 · Cited by 826
A Crash Course in Good and Bad Controls
Cinelli, C., Forney, A., & Pearl, J. (2020) · Sociological Methods & Research · doi:10.1177/00491241221099552 · Cited by 789
“…fi elds, such as sociology (Elwert, 2013; Morgan and Winship, 2015), economics (Hünermund and Bareinboim, 2019; Cunningham, 2021), psychology (Rohrer, 2018), epidemiology (Greenland, Pearl and Robins, 1999; Hernán and Robins, 2020) and statistics (Pearl et al., 2009; Pearl, Glymour and…”
performance: Assessment of Regression Models Performance
Lüdecke, D., Makowski, D., Ben‐Shachar, M. S., Patil, I., Waggoner, P., Wiernik, B. M., & Thériault, R. (2019) · doi:10.32614/cran.package.performance · Cited by 512
Metadata
- Authors
- Rohrer, J. M.
- Affiliations
- German Institute for Economic Research (DE)
- Corresponding author
- Rohrer, Julia M.
- ORCID
- Rohrer, Julia M.
- Venue
- Advances in Methods and Practices in Psychological Science
- Publisher
- SAGE Publications
- Published
- 2018-01-29
- Volume
- 1
- Issue
- 1
- Pages
- 27-42
- ISSN
- 2515-2459 (ISSN-L)
- 2515-2467
- OpenAlex
- W2776946813
- Cited by
- 1,002
- References
- 47
- Citations by year
- 2026: 124 · 2025: 145 · 2024: 150 · 2023: 147 · 2022: 122 · 2021: 132 · 2020: 137 · 2019: 31 · 2018: 14
- FWCI
- 65.35
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Advances in Methods and Practices in Psychological Science (publishedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Keywords
- causal inference, observational data, causal graphical models, confounding, third variable, particle colliders, mediation, statistical control, causal assumptions, correlation versus causation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
