The DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
The importance of covariate selection in controlling for selection bias in observational studies.
Abstract
The assumption of strongly ignorable treatment assignment is required for eliminating selection bias in observational studies. To meet this assumption, researchers often rely on a strategy of selecting covariates that they think will control for selection bias. Theory indicates that the most important covariates are those highly correlated with both the real selection process and the potential outcomes. However, when planning a study, it is rarely possible to identify such covariates with certainty. In this article, we report on an extensive reanalysis of a within-study comparison that contrasts a randomized experiment and a quasi-experiment. Various covariate sets were used to adjust for initial group differences in the quasi-experiment that was characterized by self-selection into treatment. The adjusted effect sizes were then compared with the experimental ones to identify which individual covariates, and which conceptually grouped sets of covariates, were responsible for the high degree of bias reduction achieved in the adjusted quasi-experiment. Such results provide strong clues about preferred strategies for identifying the covariates most likely to reduce bias when planning a study and when the true selection process is not known.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Empirical examples exist where controlling for pre-treatment variables demonstrably increases rather than decreases bias.”
From Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable (2014) · cites this paper as Steiner et al. 2010
In the citing paper
“For example, Steiner et al. (2010) report a within-study comparison between an observational study and an experimental benchmark of the effect of an educational intervention on student test scores. In the observational study, the bias in the estimated effect of treatment on math scores increased by about 30% when controlling only for pre-treatment psychological disposition or only for pre”
From Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable (2014) · cites this paper as Steiner et al. 2010
In the citing paper
“Furthermore, they found that although controlling for additional pre-treatment variables generally reduced bias, sometimes including additional controls increased bias.”
From Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable (2014) · cites this paper as Steiner et al. 2010
In the citing paper
“If the pre-treatment collider is also a confounder, then conditioning on it will generally, but not always, remove more bias than it induces (Greenland 2003), especially if the analysis further conditions on many other pre-treatment variables (Steiner et al. 2010).”
From Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable (2014) · cites this paper as Steiner et al. 2010
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
(2002) · Cited by 13,211
Self-citationCausality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
The development of markers for the Big-Five factor structure
Goldberg, L. R. (1992) · Psychological Assessment · doi:10.1037/1040-3590.4.1.26 · Cited by 5,530
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395
Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation
Rubin, D. B. (2001) · Health Services and Outcomes Research Methodology · doi:10.1023/a:1020363010465 · Cited by 2,228 · cited 1×
“After PS-adjustment, standardized mean differences d should be close to zero, and variance ratios v should be close to one (Rubin, 2001).”
Background
Cited by
Sorted by citations
Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable
Elwert, F., & Winship, C. (2014) · Annual Review of Sociology · doi:10.1146/annurev-soc-071913-043455 · Cited by 1,168
Predictors of early growth in academic achievement: the head-toes-knees-shoulders task
McClelland, M. M., Cameron, C. E., Duncan, R., Bowles, R. P., Acock, A. C., Miao, A., & Pratt, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00599 · Cited by 597
Barker, T. H., Stone, J. C., Sears, K., Klugar, M., Leonardi-Bee, J., Tufanaru, C., Aromataris, E., & Munn, Z. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-22-00125 · Cited by 471
Frank, K. A., Maroulis, S. J., Duong, M. Q., & Kelcey, B. M. (2013) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373713493129 · Cited by 406
Using the Propensity Score Method to Estimate Causal Effects
Li, M. (2012) · Organizational Research Methods · doi:10.1177/1094428112447816 · Cited by 382
Metadata
- Authors
- Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H.
- Affiliations
- Northwestern University (US)
- Venue
- Psychological Methods
- Published
- 2010
- Volume
- 15
- Issue
- 3
- Pages
- 250-267
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2168206775
- Cited by
- 490
- References
- 55
- Citations by year
- 2026: 23 · 2025: 28 · 2024: 34 · 2023: 26 · 2022: 26 · 2021: 53 · 2020: 32 · 2019: 20 · 2018: 37 · 2017: 36 · 2016: 38 · 2015: 28 · 2014: 31 · 2013: 32 · 2012: 19
- FWCI
- 18.95
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Funders
- US Department of Education (R305U070003)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- covariate selection, selection bias, observational studies, quasi-experiments, self-selection, randomized experiments, bias mitigation
- MeSH terms
- Data Interpretation, Statistical, Epidemiologic Methods, Humans, Mathematics, Random Allocation, Models, Statistical, Selection Bias, Causality, Patient Selection, Behavioral Research, Propensity Score
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
