Noncollapsibility and its role in quantifying confounding bias in logistic regression
Open accessAbstract
Background: Confounding bias is a common concern in epidemiological research. Its presence is often determined by comparing exposure effects between univariable- and multivariable regression models, using an arbitrary threshold of a 10% difference to indicate confounding bias. However, many clinical researchers are not aware that the use of this change-in-estimate criterion may lead to wrong conclusions when applied to logistic regression coefficients. This is due to a statistical phenomenon called noncollapsibility, which manifests itself in logistic regression models. This paper aims to clarify the role of noncollapsibility in logistic regression and to provide guidance in determining the presence of confounding bias. Methods: A Monte Carlo simulation study was designed to uncover patterns of confounding bias and noncollapsibility effects in logistic regression. An empirical data example was used to illustrate the inability of the change-in-estimate criterion to distinguish confounding bias from noncollapsibility effects. Results: The simulation study showed that, depending on the sign and magnitude of the confounding bias and the noncollapsibility effect, the difference between the effect estimates from univariable- and multivariable regression models may underestimate or overestimate the magnitude of the confounding bias. Because of the noncollapsibility effect, multivariable regression analysis and inverse probability weighting provided different but valid estimates of the confounder-adjusted exposure effect. In our data example, confounding bias was underestimated by the change in estimate due to the presence of a noncollapsibility effect. Conclusion: In logistic regression, the difference between the univariable- and multivariable effect estimate might not only reflect confounding bias but also a noncollapsibility effect. Ideally, the set of confounders is determined at the study design phase and based on subject matter knowledge. To quantify confounding bias, one could compare the unadjusted exposure effect estimate and the estimate from an inverse probability weighted model.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Karlson-Holm-Breen method controls for non-collapsibility bias, thereby providing unbiased estimates of mediation effects.
“The Karlson-Holm-Breen method controls for this non-collapsibility effect, thereby providing unbiased estimates of mediation effects.”
From Social networking site use, depressive and anxiety symptoms in adolescents: evidence from a longitudinal cohort study (SCAMP) (2026) · cites this paper as [36]
The Background explains how noncollapsibility can distort comparisons of logistic-regression coefficients, but does not say that the Karlson–Holm–Breen method corrects it or yields unbiased mediation estimates. Searches for that method and for mediation effects found it only in the References, not in a passage assessing its performance. Page numbers are unavailable in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 41 · Cited by 61
Research works printed but not linked: 32
- [1]Austin Peter C An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies Multivar Behav Res 2011 46 3 399 424 10.1080/00273171.2011.568786 PMC3144483 21818162 · DOI · Paper page
- [3]Samuels ML Matching and design efficiency in epidemiological studies Biometrika 1981 68 3 577 588 10.1093/biomet/68.3.577 · DOI
- [4]Miettinen OS Cook EF Confounding: essence and detection Am J Epidemiol 1981 114 4 593 603 10.1093/oxfordjournals.aje.a113225 7304589 · DOI
- [5]Greenland S Robins JM Identifiability, exchangeability, and epidemiological confounding Int J Epidemiol 1986 15 3 413 419 10.1093/ije/15.3.413 3771081 · DOI · Paper page
- [6]Kleinbaum DG, Sullivan KM, Barker ND. A pocket guide to epidemiology. Springer Science + Business Media, LLC; 2007.
- [7]Pang M Kaufman JS Platt RW Studying noncollapsibility of the odds ratio with marginal structural and logistic regression models Stat Methods Med Res 2016 25 5 1925 1937 10.1177/0962280213505804 24108272 · DOI
- [8]Mood C Logistic regression: why we cannot do what we think we can do, and what we can do about it Eur Sociol Rev 2009 26 1 67 82 10.1093/esr/jcp006 · DOI · Paper page
- [9]Janes H Dominici F Zeger S On quantifying the magnitude of confounding Biostatistics 2010 11 3 572 582 10.1093/biostatistics/kxq007 20203259 PMC2883302 · DOI
- [10]Cramer JS Robustness of logit analysis: unobserved heterogeneity and mis-specified disturbances Oxford Bull Econ Stat 2007 69 4 545 555 10.1111/j.1468-0084.2007.00445.x · DOI
- [11]MacKinnon DP, Luecken LJ. How and for whom? Mediation and moderation in health psychology. American Psychological Association; 2008. p. S99-S100. PMC2821200 18377161
- [12]Karlson KB Holm A Breen R Comparing regression coefficients between same-sample nested models using logit and probit: a new method Sociol Methodol 2012 42 1 286 313 10.1177/0081175012444861 · DOI
- [16]Hernan MA Clayton D Keiding N The Simpson's paradox unraveled Int J Epidemiol 2011 40 3 780 785 10.1093/ije/dyr041 21454324 PMC3147074 · DOI
- [17]Neuhaus JM Jewell NP A geometric approach to assess bias due to omitted covariates in generalized linear models Biometrika 1993 80 4 807 815 10.1093/biomet/80.4.807 · DOI
- [18]Lee PH Is a cutoff of 10% appropriate for the change-in-estimate criterion of confounder identification? J Epidemiol 2014 24 2 161 167 10.2188/jea.JE20130062 24317343 PMC3983286 · DOI
- [19]Budtz-Jorgensen E Keiding N Grandjean P Weihe P Confounder selection in environmental epidemiology: assessment of health effects of prenatal mercury exposure Ann Epidemiol 2007 17 1 27 35 10.1016/j.annepidem.2006.05.007 17027287 · DOI
- [20]Heinze G Juni P An overview of the objectives of and the approaches to propensity score analyses Eur Heart J 2011 32 14 1704 1708 10.1093/eurheartj/ehr031 21362706 · DOI · Paper page
- [21]Brookhart MA Wyss R Layton JB Sturmer T Propensity score methods for confounding control in nonexperimental research Circ Cardiovasc Qual Outcomes 2013 6 5 604 611 10.1161/CIRCOUTCOMES.113.000359 24021692 PMC4032088 · DOI
- [22]Robins JM Hernan MA Brumback B Marginal structural models and causal inference in epidemiology Epidemiology 2000 11 5 550 560 10.1097/00001648-200009000-00011 10955408 · DOI · Paper page
- [23]Rosenbaum PR Rubin DB The central role of the propensity score in observational studies for causal effects Biometrika 1983 70 41 55 10.1093/biomet/70.1.41 · DOI · Paper page
- [24]Hirano K, Imbens GW. The propensity score with continuous treatments. In: Gelmanand A, Meng X-L, editors. Applied Bayesian modeling and causal inference from incomplete-data perspectives. John Wiley & Sons, Ltd; 2004. p. 73–84.
- [25]Imai K van Dyk DA Causal inference with general treatment Regimes J Am Stat Assoc 2004 99 467 854 866 10.1198/016214504000001187 · DOI
- [26]Xu S Ross C Raebel MA Shetterly S Blanchette C Smith D Use of stabilized inverse propensity scores as weights to directly estimate relative risk and its confidence intervals Value Health 2010 13 2 273 277 10.1111/j.1524-4733.2009.00671.x 19912596 PMC4351790 · DOI
- [27]Martens EP Pestman WR de Boer A Belitser SV Klungel OH Systematic differences in treatment effect estimates between propensity score methods and logistic regression Int J Epidemiol 2008 37 5 1142 1147 10.1093/ije/dyn079 18453634 · DOI
- [31]Mansournia MA Greenland S The relation of collapsibility and confounding to faithfulness and stability Epidemiology 2015 26 466 472 10.1097/EDE.0000000000000291 25867114 · DOI
- [33]Wijnstok NJ Hoekstra T van Mechelen W Kemper HC Twisk JW Cohort profile: the Amsterdam Growth and Health Longitudinal Study Int J Epidemiol 2013 42 2 422 429 10.1093/ije/dys028 22434862 · DOI
- [36]Shrier I Platt RW Reducing bias through directed acyclic graphs BMC Med Res Methodol 2008 8 70 10.1186/1471-2288-8-70 18973665 PMC2601045 · DOI
- [38]Daniel Rhian Zhang Jingjing Farewell Daniel Making apples from oranges: comparing noncollapsible effect estimators and their standard errors after adjustment for different covariate sets Biomet J 2020 63 3 528 557 10.1002/bimj.201900297 PMC7986756 33314251 · DOI · Paper page
- [39]Breen Richard Karlson Kristian Bernt Holm Anders Interpreting and understanding logits, probits, and other nonlinear probability models Annu Rev Sociol 2018 44 1 39 54 10.1146/annurev-soc-073117-041429 · DOI · Paper page
- [40]Burgess Stephen Estimating and contextualizing the attenuation of odds ratios due to non collapsibility Communications in Statistics - Theory and Methods 2016 46 2 786 804 10.1080/03610926.2015.1006778 · DOI
- [41]Snowden JM Rose S Mortimer KM Implementation of G-computation on a simulated data set: demonstration of a causal inference technique Am J Epidemiol 2011 173 7 731 738 10.1093/aje/kwq472 21415029 PMC3105284 · DOI
- [42]Zhang Z Estimating a marginal causal odds ratio subject to confounding Communications in Statistics - Theory and Methods 2008 38 3 309 321 10.1080/03610920802200076 · DOI
- [43]Karlson KB, Popham F, Holm A. marginal and conditional confounding using logits. Sociol Methods Re. 2021. 10.1177/0049124121995548. PMC7615235 37873547 · DOI
- Authors
- Schuster, N. A., Twisk, J. W. R., ter Riet, G., Heymans, M. W., & Rijnhart, J. J. M.
- Affiliations
- Amsterdam Public Health (NL)
- Corresponding author
- Schuster, Noah A.
- Venue
- BMC Medical Research Methodology
- Published
- 2021-07-05
- Volume
- 21
- Issue
- 1
- Pages
- 136-136
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC8259440
- OpenAlex
- W3180543148
- Cited by
- 61
- References
- 41
- Citations by year
- 2026: 17 · 2025: 10 · 2024: 15 · 2023: 13 · 2022: 6
- FWCI
- 6.34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
- Pure Amsterdam UMC (acceptedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- confounding, logistic regression, inverse probability weighting, confounder selection
- MeSH terms
- Humans, Probability, Research Design, Bias, Logistic Models, Epidemiologic Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
