CoChat
Sign Up
Cited by 513Advanced Causal Inference TechniquesStatistical Methods in Clinical TrialsStatistical Methods in Epidemiology

The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.

Double-adjustment in propensity score matching analysis: choosing a threshold for considering residual imbalance

Open access
Nguyen, Tri-Long, Collins, Gary S., Spence, Jessica, Daurès, Jean-Pierre, Devereaux, P. J., Landais, Paul, Le Manach, Yannick·2017·BMC Medical Research Methodology
DOI 10.1186/s12874-017-0338-0PMC PMC5408373
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Double-adjustment can be used to remove confounding if imbalance exists after propensity score (PS) matching. However, it is not always possible to include all covariates in adjustment. We aimed to find the optimal imbalance threshold for entering covariates into regression. METHODS: We conducted a series of Monte Carlo simulations on virtual populations of 5,000 subjects. We performed PS 1:1 nearest-neighbor matching on each sample. We calculated standardized mean differences across groups to detect any remaining imbalance in the matched samples. We examined 25 thresholds (from 0.01 to 0.25, stepwise 0.01) for considering residual imbalance. The treatment effect was estimated using logistic regression that contained only those covariates considered to be unbalanced by these thresholds. RESULTS: We showed that regression adjustment could dramatically remove residual confounding bias when it included all of the covariates with a standardized difference greater than 0.10. The additional benefit was negligible when we also adjusted for covariates with less imbalance. We found that the mean squared error of the estimates was minimized under the same conditions. CONCLUSION: If covariate balance is not achieved, we recommend reiterating PS modeling until standardized differences below 0.10 are achieved on most covariates. In case of remaining imbalance, a double adjustment might be worth considering.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12874-017-0338-0

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Covariate balance diagnostics after nearest-neighbor matching in (a) a linear, additive scenario and (b) a non-linear, non-additive scenario. SMD, standardized absolute mean difference
Fig. 1 Covariate balance diagnostics after nearest-neighbor matching in (a) a linear, additive scenario and (b) a non-linear, non-additive scenario. SMD, standardized absolute mean differenceCC BY · PubMed Central
Fig. 2 Estimated average treatment effect in the treated (ATT) on the absolute risk difference scale (median with 2.5th and 97.5th percentiles), in (a) the linear, additive scenario and (b) the non-linear, non-additive scenario. The double-robust estimator was adjusted for unbalanced covariates using standardized absolute mean difference (SMD) thresholds
Fig. 2 Estimated average treatment effect in the treated (ATT) on the absolute risk difference scale (median with 2.5th and 97.5th percentiles), in (a) the linear, additive scenario and (b) the non-linear, non-additive scenario. The double-robust estimator was adjusted for unbalanced covariates using standardized absolute mean difference (SMD) thresholdsCC BY · PubMed Central
Fig. 3 Percentage of biased estimates according to the estimator in (a and c) the linear, additive scenario and (b and d) the non-linear, non-additive scenario. The double-robust estimator was adjusted for unbalanced covariates using standardized absolute mean difference (SMD) thresholds
Fig. 3 Percentage of biased estimates according to the estimator in (a and c) the linear, additive scenario and (b and d) the non-linear, non-additive scenario. The double-robust estimator was adjusted for unbalanced covariates using standardized absolute mean difference (SMD) thresholdsCC BY · PubMed Central
Fig. 4 Mean squared error according to the estimator in (a) the linear, additive scenario, and (b) the non-linear and non-additive scenario. The double-robust estimator was adjusted for unbalanced covariates using standardized absolute mean difference (SMD) thresholds
Fig. 4 Mean squared error according to the estimator in (a) the linear, additive scenario, and (b) the non-linear and non-additive scenario. The double-robust estimator was adjusted for unbalanced covariates using standardized absolute mean difference (SMD) thresholdsCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 1

  1. Ho DE, Imai K, King G, Stuart EA. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit Anal. 2006;15(3):199–236.

Cited by

Sorted by citations

See all 513

Related

See all

Metadata

Authors
Nguyen, T.-L., Collins, G. S., Spence, J., Daurès, J.-P., Devereaux, P. J., Landais, P., & Le Manach, Y.
Affiliations
  • Université de Montpellier (FR)
  • Population Health Research Institute (CA)
ORCID
Venue
BMC Medical Research Methodology
Publisher
Springer Science and Business Media LLC
Published
2017-04-28
Volume
17
Issue
1
Pages
78-78
ISSN
1471-2288 (ISSN-L)
DOI
10.1186/s12874-017-0338-0
PMC
PMC5408373
OpenAlex
W2609185034
Cited by
512
References
35
Citations by year
2026: 65 · 2025: 79 · 2024: 76 · 2023: 76 · 2022: 86 · 2021: 61 · 2020: 32 · 2019: 19 · 2018: 12 · 2017: 5
FWCI
12.93
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (publishedVersion, other-oa)
  • HAL (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
None
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
propensity score matching, standardized mean difference, covariate balance, regression adjustment, confounding, nearest neighbor matching, treatment effect estimation, mean squared error
MeSH terms
Algorithms, Computer Simulation, Humans, Monte Carlo Method, Models, Statistical, Bias, Logistic Models, Propensity Score

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review