Bias Amplification and Bias Unmasking
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
In the analysis of causal effects in non-experimental studies, conditioning on observable covariates is one way to try to reduce unobserved confounder bias. However, a developing literature has shown that conditioning on certain covariates may increase bias, and the mechanisms underlying this phenomenon have not been fully explored. We add to the literature on bias-increasing covariates by first introducing a way to decompose omitted variable bias into three constituent parts: bias due to an unobserved confounder, bias due to excluding observed covariates, and bias due to amplification. This leads to two important findings. Although instruments have been the primary focus of the bias amplification literature to date, we identify the fact that the popular approach of adding group fixed effects can lead to bias amplification as well. This is an important finding because many practitioners think that fixed effects are a convenient way to account for any and all group-level confounding and are at worst harmless. The second finding introduces the concept of bias unmasking and shows how it can be even more insidious than bias amplification in some cases. After introducing these new results analytically, we use constructed observational placebo studies to illustrate bias amplification and bias unmasking with real data. Finally, we propose a way to add bias decomposition information to graphical displays for sensitivity analysis to help practitioners think through the potential for bias amplification and bias unmasking in actual applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation
Rubin, D. B. (2001) · Health Services and Outcomes Research Methodology · doi:10.1023/a:1020363010465 · Cited by 2,228 · cited 1×
“One recommendation has been that conditioning on more, rather than fewer, available (pretreatment) covariates is the best way to minimize bias associated with unobserved sources of heterogeneity (Rosenbaum 2002; Rubin 2002).”
BackgroundOveradjustment bias and unnecessary adjustment in epidemiologic studies.
Schisterman, E. F., Cole, S. R., & Platt, R. W. (2009) · Epidemiology · doi:10.1097/ede.0b013e3181a819a1 · Cited by 2,096 · cited 2×
“Bias inducers include posttreatment variables such as mediators and colliders (Cole et al. 2010; Pearl 2000; Schisterman et al. 2009), and a particular group of pretreatment covariates (pretreatment colliders that lead to M-bias or butterfly-bias) (Ding and Miratrix 2014; Sjlander 2009; Pearl 2009).”
BackgroundTotal, Direct, and Indirect Effects in Logit and Probit Models
Breen, R., Karlson, K. B., & Holm, A. (2013) · Sociological Methods & Research · doi:10.1177/0049124113494572 · Cited by 818
Quantifying Biases in Causal Models: Classical Confounding vs Collider-Stratification Bias
Greenland, S. (2003) · Epidemiology · doi:10.1097/01.ede.0000042804.12056.6c · Cited by 810 · cited 2×
“Such bias inducers may not be troublesome in practice, however, either because they can be identified for exclusion, as is sometimes the case for posttreatment variables, or because the bias they induce tends to be small (Greenland 2002; Liu et al. 2012; Ding and Miratrix 2014).”
BackgroundIllustrating bias due to conditioning on a collider
Cole, S. R., Platt, R. W., Schisterman, E. F., Chu, H., Westreich, D., Richardson, D., & Poole, C. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyp334 · Cited by 796
Sibling Comparison Designs: Bias From Non-Shared Confounders and Measurement Error
Frisell, T., Öberg, S., Kuja-Halkola, R., & Sjölander, A. (2012) · Epidemiology · doi:10.1097/ede.0b013e31825fa230 · Cited by 551
The Phantom Menace: Omitted Variable Bias in Econometric Research
Clarke, K. A. (2005) · Conflict Management and Peace Science · doi:10.1080/07388940500339183 · Cited by 484 · cited 4×
“There is even some controversy within the controversy: in political science, Clarke (2005) contends that researchers are encouraged to include as many covariates as possible, without regard to the potential for increased bias; relatedly, in most intermediate econometrics texts, the choice between…”
BackgroundMethodResultInfluentialSobel, M. E. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000636 · Cited by 468
Do Instrumental Variables Belong in Propensity Scores?
Bhattacharya, J., & Vogt, W. (2007) · doi:10.3386/t0343 · Cited by 81
Cited by
Principles of confounder selection
VanderWeele, T. J. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00494-6 · Cited by 1,535
Making sense of sensitivity: extending omitted variable bias
Cinelli, C., & Hazlett, C. (2020) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12348 · Cited by 1,088
Information equivalence in survey experiments
Dafoe, A., Zhang, B., & Caughey, D. (2018) · Political Analysis · doi:10.1017/pan.2018.9 · Cited by 356
“Further, as in observational studies (Middleton et al. 2016), controlling for even pretreatment background characteristics can amplify bias in survey-experimental estimands.”
BackgroundDirectional Bias Amplification
Wang, A., & Russakovsky, O. (2021) · arXiv · doi:10.48550/arxiv.2102.12594 · Cited by 30
“Bias amplification is also studied in the causal statistics literature (Bhattacharya & Vogt, 2007; Wooldridge, 2016; Pearl, 2010; 2011; Middleton et al., 2016).”
Background
Metadata
- Authors
- Middleton, J. A., Scott, M. A., Diakow, R., & Hill, J. L.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2016
- Volume
- 24
- Issue
- 3
- Pages
- 307-323
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2483944069
- Cited by
- 59
- References
- 11
- FWCI
- 5.52
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.