Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable

Gangl and Ziefle's selectivity-corrected analysis of the motherhood wage penalty finds the underestimation pattern.

“This is exactly what Gangl & Ziefle’s (2009) selectivity-corrected analysis of the motherhood wage penalty finds.”

• cites Gangl & Ziefle 2009 sourceSupported

Over time, the number of cases available for analysis in prospective longitudinal studies decreases, sometimes drastically.

“Over time, the number of cases available for analysis decreases, sometimes drastically so (e.g., Behr et al. 2005, Alderman et al. 2001).”

• cites Behr et al. 2005 sourceNot checked — no full text of the source
• cites Alderman et al. 2001 sourceSupported

• AssumptionRests on an assumption: Most prospective longitudinal studies experience attrition over time.

Shalizi and Thomas showed that latent homophily bias is an example of endogenous selection bias, with the social tie as a pre-treatment collider.

“Shalizi & Thomas (2011) recently showed that latent homophily bias constitutes a previously unrecognized example of endogenous selection bias, where the social tie itself plays the role of the pre-treatment collider.”

• cites Shalizi & Thomas 2011 sourceSupported

If the analyst assumes that controlling for some combination of observed pre-treatment variables is sufficient to identify the total causal effect, then it is sufficient to control for all variables that cause treatment or outcome or both.

“If the analyst is willing to assume that controlling for some combination of observed pre-treatment variables is sufficient to identify the total causal effect of treatment, then it is sufficient to control for all variables that (directly or indirectly) cause treatment or outcome or both (VanderWeele & Shpitser 2011).”

• cites VanderWeele & Shpitser 2011 sourceSupported

• AssumptionThe authors state this as an assumption: The analyst assumes that controlling for some combination of observed pre-treatment variables is sufficient to identify the total causal effect of treatment.

Sociological surveys of sample selection bias tend to focus on cases where selection is affected by treatment or outcome, i.e., after treatment.

“Surveys of sample selection bias in sociology tend to emphasize examples where selection is affected by the treatment or the outcome, i.e., where sample selection occurs after treatment”

• cites Berk 1983 sourceNot checked — no full text of the source
• cites Winship & Mare 1992 sourceNot checked — no full text of the source
• cites Stolzenberg & Relles 1997 sourceNot checked — no full text of the source

All nonparametric identification problems can be classified as overcontrol bias, confounding bias, or endogenous selection bias.

“all nonparametric identification problems can be classified as one of three underlying problems: overcontrol bias, confounding bias, and endogenous selection bias.”

Identification analysis needs a theory of how the data were generated.

“Identification analysis requires a theory (i.e., a model or assumptions) of how the data were generated.”

Identification with ideal data is a necessary precondition for unbiased estimation with real data.

“Identification (assuming ideal data) is a necessary precondition for unbiased estimation (with real data).”

There is currently no universal solution for estimating causal effects when endogenous selection is present.

“there is, at present, no universal solution to estimating causal effects in the presence of endogenous selection”

• cites Stolzenberg & Relles 1997 sourceNot checked — no full text of the source

The analyst's theory of data generation can never be fully tested empirically.

“the analyst’s theory of data generation can never be fully tested empirically”

• cites Robins & Wasserman 1999Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Causal effects must be inferred from observed associations, which mix the causal effect with noncausal components.

“causal effects have to be inferred from observed associations, which are generally a mixture of the desired causal effect and various noncausal, or spurious, components.”

Restricting a sample to employed women biases estimates of motherhood's effect on wages.

“estimates for the effect of motherhood on wages will be biased if the sample is restricted to employed women”

• cites Gronau 1974 sourceNot checked — no full text of the source
• cites Heckman 1974 sourceNot checked — no full text of the source

Endogenous selection bias is broader than after-treatment sample selection because it can arise from conditioning on variables even when no observations are discarded.

“Second, bias can occur from simple conditioning (e.g., controlling or stratifying) on certain variables even if no observations are discarded from the population (i.e., if no sample is selected).”

Restricting a sample to low earners biases estimates of education's effect on wages.

“estimates of the effect of education on wages will be biased if the sample is restricted to low earners”

• cites Hausman & Wise 1977 sourceNot checked — no full text of the source

Causal effects cannot be observed directly.

“causal effects can never be observed directly”

• cites Holland 1986 sourceNot checked — no full text of the source

Endogenous selection captures the shared structure of many biases known by different names, such as selective nonresponse, ascertainment bias, attrition bias, informative censoring, Heckman selection bias, sample selection bias, and homophily bias.

“Endogenous selection captures the common structure of a large number of biases usually known by different names, including selective nonresponse, ascertainment bias, attrition bias, informative censoring, Heckman selection bias, sample selection bias, homophily bias, and others.”

Bias from endogenous selection is a major issue in causal inference.

“Endogenous selection bias is a central problem for causal inference.”

The common advice not to select on the dependent variable is too narrow.

“the commonplace advice not to select (i.e., condition) on the dependent variable of an analysis is too narrow.”

DAGs reveal all marginal and conditional associations and independences implied by a qualitative causal model.

“The power of DAGs lies in their ability to reveal all marginal and conditional associations and independences implied by a qualitative causal model”

• cites Pearl 2009 sourceNot checked — no full text of the source

The endogenous selection problem of mediation analysis is of particular concern because conventional efforts to control unobserved confounding focus on confounders of the main treatment but not on confounders of the mediator.

“The endogenous selection problem of mediation analysis is of particular concern for current empirical practice in sociology because conventional efforts to control for unobserved confounding typically focus on the confounders of the main treatment but not on the confounders of the mediator.”

If motherhood decreases employment chances and higher offer wages increase acceptance chances for all women, then employed mothers must on average have received higher wage offers than employed childless women.

“If, for example, motherhood decreases the chances of employment (by increasing reservation wages), and if higher offer wages increase the chances of accepting employment for all women, then employed mothers must on average have received higher wage offers than employed childless women.”

Steiner et al. found that in an observational study of an educational intervention, bias in the estimated effect on math scores increased by about 30% when controlling only for pre-treatment psychological disposition or only for pre-treatment vocabulary test scores.

“For example, Steiner et al. (2010) report a within-study comparison between an observational study and an experimental benchmark of the effect of an educational intervention on student test scores. In the observational study, the bias in the estimated effect of treatment on math scores increased by about 30% when controlling only for pre-treatment psychological disposition or only for pre”

• cites Steiner et al. 2010 sourceNot checked — no full text of the source

Socially connected individuals may exhibit similar behaviors because similar individuals form ties rather than because one influences the other.

“A classic problem of causal inference in social networks is that socially connected individuals may exhibit similar behaviors not because one individual influences the other (causation) but because individuals who are similar tend to form social ties with each other (homophily) (Farr 1858).”

• cites Farr 1858Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Studying the causes of success in a sample selected for success invites endogenous selection bias.

“studying the causes of success in a sample selected for success invites endogenous selection bias.”

If tie formation or dissolution is affected by unobserved variables associated with the treatment in one individual and the outcome in another, searching for interpersonal effects induces a spurious association between individuals.

“In sum, if tie formation or tie dissolution is affected by unobserved variables that are associated with, respectively, the treatment variable in one individual and the outcome variable in another individual, then searching for interpersonal effects will induce a spurious association between individuals in the network, i.e., endogenous selection bias.”

Adding arrows to a given set of variables never helps nonparametric identification.

“Adding arrows to a given set of variables never helps nonparametric identification”

• cites Pearl 2009 sourceNot checked — no full text of the source

Empirical examples exist where controlling for pre-treatment variables demonstrably increases rather than decreases bias.

“Empirical examples exist where controlling for pre-treatment variables demonstrably increases rather than decreases bias.”

• cites Steiner et al. 2010 sourceNot checked — no full text of the source

The magnitude of the association between motherhood and offer wages after conditioning on reservation wage and employment is biased for the true causal effect.

“Therefore, the magnitude of the association between M and WO after conditioning on WR and E would be biased for the true magnitude of the causal effect.”

The endogenous selection bias in the Rolling Stone example is ascertainment bias because control albums are sampled on exclusion from the Rolling Stone 500 and commercial success.

“The endogenous selection bias in the present example is ascertainment bias because control albums are sampled as a function of (a) exclusion from the Rolling Stone 500 and (b) commercial success.”

An empirical analysis of motherhood on offer wages restricted to working women would underestimate the motherhood wage penalty.

“Consequently, an empirical analysis of motherhood on offer wages that is restricted to working women would underestimate the motherhood wage penalty.”

The sign and magnitude of endogenous selection bias cannot generally be predicted without additional assumptions.

“sign and magnitude of endogenous selection bias cannot generally be predicted without additional assumptions”

• cites Hausman & Wise 1981Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Berk 1983 sourceNot checked — no full text of the source
• cites VanderWeele & Robins 2009bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Conditioning on the common outcome of two variables, a collider, induces a spurious association between them.

“Conditioning on the common outcome of two variables (i.e., a collider) induces a spurious association between them.”

Causal information can sometimes be recovered despite direct selection on the outcome when certain constraints are imposed on the data.

“We note that the recovery of causal information despite direct selection on the outcome is sometimes possible if certain constraints can be imposed on the data.”

• cites Rothman et al. 2008 sourceNot checked — no full text of the source
• cites Manski 2003 sourceNot checked — no full text of the source

Heckman selection bias can be explicated as endogenous selection bias.

“Heckman selection bias—perhaps the best-known selection bias in the social sciences—can be explicated as endogenous selection bias.”

Common approaches to estimating direct and indirect effects are highly susceptible to endogenous selection bias.

“Common approaches to estimating direct and indirect effects (Baron & Kenny 1986) are highly susceptible to endogenous selection bias.”

• cites Baron & Kenny 1986 sourceNot checked — no full text of the source

In the motherhood wage example, employment is a collider on the path from motherhood to offer wages via reservation wage.

“Therefore, employment is a collider on the path M→WR→E←WO between motherhood and offer wages.”

• AssumptionThe authors state this as an assumption: Motherhood affects a woman's reservation wage.
• AssumptionRests on an assumption: A woman accepts employment only if the wage offer meets or exceeds her reservation wage.

If motherhood and offer wages are not associated after conditioning on reservation wage and employment, then there is no causal effect of motherhood on offer wages.

“If M and WO are not associated after conditioning on WR and E, then there is no causal effect of M on WO as in Figure 8a.”

If a treatment affects an outcome, the outcome is a collider on the path between the treatment and the outcome's error term.

“If a treatment, T, has an effect on the outcome, Y, T→Y, then the outcome is a collider variable on the path between the treatment and the outcome’s error term, U, T→Y←U.”

Endogenous selection bias can originate from conditioning on a collider at any point in a causal process.

“Endogenous selection bias may originate from conditioning on a collider at any point in a causal process.”

In the dyad example, conditioning on the existence of a social tie opens a noncausal path between one person's prior civic engagement and the other's later civic engagement, producing an association even if no causal effect exists.

“The problem, then, is that the very act of computing the association between friends means that the analysis conditions on the existence of the social tie, Fi,j = 1. Conditioning on Fi,j opens a non-causal path between treatment and outcome, Yi(t)←Ui→Fi,j←Uj→Yj(t+1), which results in an association between i’s and j’s civic engagement even if one does not cause the other.”

• AssumptionThe authors state this as an assumption: Civic engagement does not spread between friends.
• AssumptionThe authors state this as an assumption: Each person's civic engagement is caused by their own characteristics, which are at least partially unobserved.
• AssumptionThe authors state this as an assumption: Tie formation is homophilous, such that altruistic people form preferential attachments.

Attrition bias is nothing other than endogenous selection bias.

“Attrition bias thus is nothing other than endogenous selection bias.”

Under the assumption that causal effects are in the same direction for all women, the direction of the bias can be predicted.

“One can predict the direction of the bias under additional assumptions, specifically that the causal effects in Figure 8 are in the same direction for all women (VanderWeele & Robins 2009b).”

• cites VanderWeele & Robins 2009bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Absent sampling variation, all observable associations come from chains, forks, and inverted forks.

“Absent sampling variation (i.e., chance), all observable associations originate from just three elementary configurations: chains (A→B, A→C→B, etc.), forks (A←C→B), and inverted forks (A→C←B)”

• cites Pearl 1988, 2009Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Verma & Pearl 1988Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Controlling for pre-treatment variables can sometimes increase rather than decrease bias.

“Controlling for pre-treatment variables can sometimes increase rather than decrease bias.”

Identifiability can be determined by checking whether conditioning on suitable observed variables blocks all noncausal paths between treatment and outcome without blocking any causal paths.

“One can then determine the identifiability of a causal effect by checking whether one can block all noncausal paths between treatment and outcome without blocking any causal paths between treatment and outcome by conditioning on a suitable set of observed variables.”

Violating the rule against sampling on both treatment and outcome produces ascertainment bias.

“The consequence of violating this rule is known as ascertainment bias (Rothman et al. 2008).”

• cites Rothman et al. 2008 sourceNot checked — no full text of the source

Nonparametric identification of the magnitude of the causal effect of motherhood on offer wages is impossible when the effect exists, because conditioning on employment creates a noncausal path from motherhood to the error term.

“Nonparametric identification of the magnitude of the causal effect M→WO, however, is made impossible by its very existence, as seen in Figure 8b: Even if WR is observed and controlled (thus blocking the flow of association along the noncausal path M→WR→E←WO), conditioning on E would still create the noncausal path M- - -ε→WO.”

If motherhood affects offer wages, conditioning on employment also conditions on a descendant of the collider offer wages on the path from motherhood to the error term, inducing a noncausal association between motherhood and the error term.

“conditioning on E now further amounts to conditioning on the descendant of the collider WO on the path M→WO←ε. This induces a noncausal association between motherhood and the error term on offer wages, M- - -ε→WO.”

Methodologists have long warned that conditioning on an intermediate variable affected by treatment can lead to bias.

“Methodologists have long warned that conditioning on an intermediate variable that has been affected by the treatment can lead to bias (e.g., Heckman 1976; Berk 1983; Rosenbaum 1984; Holland 1988; Robins 1989, 2001, 2003; Smith 1990; Angrist & Krueger 1999; Wooldridge 2005; Sobel 2008).”

• cites Heckman 1976Not checked — no full text of the source
• cites Berk 1983 sourceNot checked — no full text of the source
• cites Rosenbaum 1984 sourceNot checked — no full text of the source

An analysis restricted to employed women would show an association between motherhood and wages even if the causal effect of motherhood on wages is zero.

“The analysis would thus indicate an association between motherhood and wages even if the causal effect of motherhood on wages is in fact zero (Figure 8a).”

Conditioning on a descendant of a collider has qualitatively the same consequence as conditioning on the collider itself.

“Conditioning on a variable that is caused by a collider (a descendant of the collider) has qualitatively the same consequence as conditioning on the collider itself”

When Q is measured after schooling onset, it is difficult to determine whether controlling for Q or not will dominate empirically without detailed knowledge of effect strengths in the DAG.

“Absent detailed knowledge of the relative strengths of all effects in the DAG—which is rarely available, especially where unobserved variables are involved—it is difficult to determine which problem will dominate empirically and hence which course of action should be taken.”

In the absence of common-cause confounding, the population odds ratio gives the true causal effect of topping the Billboard charts on inclusion in the Rolling Stone 500.

“In the absence of common-cause confounding (as encoded in Figure 7), the population odds ratio ORP = ad/bc gives the true causal effect of B on R.”

• AssumptionThe authors state this as an assumption: In the music example, there is no common-cause confounding.

In case-control studies, observations should not be sampled as a function of both the treatment and the outcome.

“A key rule in case-control studies, however, is that observations should not be sampled (ascertained) as a function of both the treatment and the outcome.”

• cites Rothman et al. 2008 sourceNot checked — no full text of the source

Nonresponse bias in retrospective studies shares the same causal structure as truncation bias, except nonresponse is better understood as conditioning on a variable affected by the outcome rather than the outcome itself.

“The same causal structure that leads to truncation bias also underlies non-response bias in retrospective studies, except that nonresponse is better understood as conditioning on a variable affected by the outcome rather than conditioning on the outcome itself.”

Conditioning on an endogenous variable (a collider), regardless of its temporal position relative to treatment and outcome, can induce new noncausal associations likely to result in biased estimates.

“A key insight is that conditioning on an endogenous variable (a collider variable), no matter where it temporally falls relative to treatment and outcome, can induce new, noncausal, associations that are likely to result in biased estimates.”

If a pre-treatment collider is also a confounder, conditioning on it generally but not always removes more bias than it induces, especially if the analysis conditions on many other pre-treatment variables.

“If the pre-treatment collider is also a confounder, then conditioning on it will generally, but not always, remove more bias than it induces (Greenland 2003), especially if the analysis further conditions on many other pre-treatment variables (Steiner et al. 2010).”

• cites Greenland 2003 sourceNot checked — no full text of the source
• cites Steiner et al. 2010 sourceNot checked — no full text of the source

Conditioning on a pre-treatment collider will usually lead to a small bias.

“Finally, conditioning on a pre-treatment collider (as in Figures 12 and 13) will usually lead to a small bias, which is why recognizing latent homophily bias as endogenous selection bias (as opposed to confounding bias) should be good news to network analysts.”

• cites Greenland 2003 sourceNot checked — no full text of the source

Conditioning on post-outcome colliders affected by treatment and outcome induces the same bias as the corresponding confounding bias if the arrows into the collider were reversed.

“First, conditioning on post-outcome colliders that are affected by treatment and outcome, T→C←Y, induces the same bias as would the corresponding confounding bias if the directions of the arrows into the collider were reversed, T←C→Y.”

• cites Greenland 2003 sourceNot checked — no full text of the source

The only way to decide whether to control for X is to decide a priori which causal model represents the data-generating process.

“Therefore, the only way to decide whether or not to control for X is to decide, on a priori grounds, which of the two causal models accurately represents the data-generating process.”

In both Figure 13a and 13b, all combinations of X, T, and Y are marginally and conditionally associated.

“In both models, all combinations of X, T, and Y are marginally and conditionally associated with each other.”

Absent strong parametric assumptions, size and sign of the bias depend on the exact DAG structure, the sizes of effects and their variation across individuals, and the distribution of the variables.

“Absent strong parametric assumptions, size and sign of the bias will depend on the exact structure of the DAG, the sizes of the effects and their variation across individuals, and the distribution of the variables (Greenland 2003).”

• cites Greenland 2003 sourceNot checked — no full text of the source

The conventional advice to condition on X in both Figure 13a and 13b is incorrect.

“But this conventional advice is incorrect.”

VanderWeele and Robins derived the sign of endogenous selection bias for DAGs with binary variables and monotonic effects with and without interactions.

“For example, VanderWeele & Robins (2007a, 2009a,b) derive the sign of endogenous selection bias for DAGs with binary variables and monotonic effects with and without interactions.”

• cites VanderWeele & Robins 2007aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites VanderWeele & Robins 2009aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites VanderWeele & Robins 2009bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

When all variables in the DAG are binary, Greenland suggests that controlling for a variable that is both a confounder and a collider likely but not necessarily removes more bias than it creates.

“When all variables in the DAG are binary, Greenland (2003) suggests that controlling for a variable that is both a confounder and a collider, as in Figure 13c, likely (but not necessarily) removes more bias than it creates.”

• cites Greenland 2003 sourceNot checked — no full text of the source

Sign and size of endogenous selection bias are difficult to predict.

“One of the remaining difficulties in dealing with endogenous selection (if it cannot be entirely avoided) is that sign and size of the bias are difficult to predict.”

If U1 and U2 are unobserved, the analyst cannot empirically distinguish Figure 13a from 13b because both have identical observational implications.

“If U1 and U2 are unobserved, the analyst cannot distinguish empirically between Figure 13a and b because both models have identical observational implications.”

Conventional intuition about size and sign of bias can break down outside linear models.

“Conventional intuition about size and sign of bias in causal systems can break down in unpredictable ways outside of the stylized assumptions of linear models.”

Selection of control variables requires a theoretical commitment to a model of data generation.

“In other words, the selection of control variables requires a theoretical commitment to a model of data generation.”

Questions about which pre-treatment variables to control cannot be settled without a theory of how the data were generated.

“An inspection of the DAGs in Figure 13 demonstrates that these questions cannot be settled without recourse to a theory of how the data were generated.”