Assessing bias: the importance of considering confounding

Abstract

Overview: Why evaluate bias? Isn’t statistical significance enough? It is common to come across a study that reports that treatment A “provided significantly better pain relief” than treatment B. When a difference in an outcome (eg, pain) between exposures (eg, treatment groups) is observed, one needs to consider whether the effect is truly because of exposure or if alternate explanations are possible. As we discussed in the previous issue of EBSJ, to evaluate the validity of a research study, factors that might distort the true association and/or influence its interpretation need to be carefully considered. This means evaluating the role of bias and considering the study’s statistical precision. In the previous issue, we discussed aspects of statistical testing and precision to explore the question of whether an effect is real or due to chance. We also considered some caveats to concluding that a “statistically significant” result is clinically meaningful. This article takes a look at the potential for bias and its impact. Bias relates to systematic sources of error which need to be considered. Why? The internal validity of a study depends greatly on the extent to which biases have been accounted for and necessary steps taken to diminish their impact. In a poor-quality study, bias may be the primary reason the results are or are not “significant” statistically! Bias may preclude finding a true effect; it may lead to an inaccurate estimate (underestimate or overestimate) of the true association between exposure and an outcome. Significance testing in itself does not take into account factors which may bias study results. Bias can be divided into three general categories: (1) selection bias; (2) information bias; and (3) confounding. This article focuses on confounding. Recognizing it and controlling for its effects are important to a study’s credibilty. What is confounding? Confounding is often referred to as a “mixing of effects” [ 1 ], [ 2 ] wherein the effects of the exposure under study on a given outcome are mixed in with the effects of an additional factor (or set of factors) resulting in a distortion of the true relationship. In a clinical trial, this can happen when the distribution of a known prognostic factor differs between groups being compared. Confounding factors may mask an actual association or, more commonly, falsely demonstrate an apparent association between the treatment and outcome when no real association between them exists. The existence of confounding variables in studies make it difficult to establish a clear causal link between treatment and outcome unless appropriate methods are used to adjust for the effect of the confounders (more on this below). Confounding variables are those that may compete with the exposure of interest (eg, treatment) in explaining the outcome of a study. The amount of association “above and beyond” that which can be explained by confounding factors provides a more appropriate estimate of the true association which is due to the exposure. General characteristics of confounders include the following: A true confounding factor is predictive of the outcome even in the absence of the exposure. Although a potential confounding factor (PCF) may be causative, it might not be. The primary requirements are that an independent relationship between the factor and the outcome exists and that the PCF not be the result of the exposure (or the outcome). In fact, many of the PCFs which often must be evaluated are proxies for variables which are complex and difficult to measure ([ Fig 1 ]). A confounding factor is also associated with the exposure being studied but is not a proxy or surrogate for the exposure. In clinical trials, confounding is often a result of unequal distribution of the potential confounding factors between treatment groups ([ Fig 2 ]). A situation that contains both numbers 1 and 2 sets the stage for potential confounding ([ Fig 3 ]). A confounder cannot be an intermediate between the exposure and the outcome. For example, the relationship between diet and coronary heart disease may be explained by measuring serum cholesterol level. Cholesterol is not a confounder because it may be a causal link between diet and coronary heart disease ([ Fig 4 ]). Fig 1 Fig 2 Fig 3 Fig 4 How does confounding influence results? An example in spine surgery research: Let’s imagine that we wanted to know if treating index osteoporotic vertebral fractures with vertebroplasty increased the risk of subsequent vertebral fractures. We evaluate 400 patients with index vertebral fractures, 200 of whom received vertebroplasty and 200 did not. After 2 years, we identified 45 subsequent fractures with the following fictitious distribution: At first glance, it looks like those who received vertebroplasty were at a much higher risk (two times the risk) compared with those who did not (this is called a “crude” estimate of the association). However, it is important to investigate whethe