The DOI matches its Crossref record; 39 of the 40 references it cites were found in a registry or an index.
Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes
Open accessAbstract
AIMS: Randomized controlled trials (RCTs) are the gold standard for assessing the efficacy of therapeutic interventions because randomization protects from biases inherent in observational studies. Propensity score (PS) methods, proposed as a potential solution to confounding of the treatment-outcome association, are widely used in observational studies of therapeutic interventions for acute coronary syndromes (ACS). We aimed to systematically assess agreement between observational studies using PS methods and RCTs on therapeutic interventions for ACS. METHODS AND RESULTS: We searched for observational studies of interventions for ACS that used PS methods to estimate treatment effects on short- or long-term mortality. Using a standardized algorithm, we matched observational studies to RCTs based on patients' characteristics, interventions, and outcomes ('topics'), and we compared estimates of treatment effect between the two designs. When multiple observational studies or RCTs were identified for the same topic, we performed a meta-analysis and used the summary relative risk for comparisons. We matched 21 observational studies investigating 17 distinct clinical topics to 63 RCTs (median = 3 RCTs per observational study) for short-term (7 topics) and long-term (10 topics) mortality. Estimates from PS analyses differed statistically significantly from randomized evidence in two instances; however, observational studies reported more extreme beneficial treatment effects compared with RCTs in 13 of 17 instances (P = 0.049). Sensitivity analyses limited to large RCTs, and using alternative meta-analysis models yielded similar results. CONCLUSION: For the treatment of ACS, observational studies using PS methods produce treatment effect estimates that are of more extreme magnitude compared with those from RCTs, although the differences are rarely statistically significant.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Such methods often yield results similar to those in controlled treatment trials (133, 134).”
From Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System (2020) · cites this paper as [133]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Heart Disease and Stroke Statistics—2023 Update: A Report From the American Heart Association
Tsao, C. W., Aday, A. W., Almarzooq, Z. I., Anderson, C. A. M., Arora, P., Avery, C. L., Baker-Smith, C. M., Beaton, A. Z., Boehme, A. K., Buxton, A. E., et al. (2023) · Circulation · doi:10.1161/cir.0000000000001123 · Cited by 8,300
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Heart Disease and Stroke Statistics—2011 Update: A Report From the American Heart Association
Roger, V. L., Go, A. S., Lloyd-Jones, D. M., Adams, R. J., Berry, J. D., Brown, T. M., Carnethon, M. R., Dai, S., de Simone, G., Ford, E. S., et al. (2011) · Circulation · doi:10.1161/cir.0b013e3182009701 · Cited by 6,149
Braunwald, E., Antman, E. M., Beasley, J. W., Califf, R. M., Cheitlin, M. D., Hochman, J. S., Jones, R. H., Kereiakes, D. J., Kupersmith, J., Levin, T. N., et al. (2000) · Circulation · doi:10.1161/01.cir.102.10.1193 · Cited by 4,475
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
A comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
What is meant by intention to treat analysis? Survey of published randomised controlled trials
Hollis, S., & Campbell, F. (1999) · BMJ · doi:10.1136/bmj.319.7211.670 · Cited by 2,160
A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003
Austin, P. C. (2007) · Statistics in Medicine · doi:10.1002/sim.3150 · Cited by 1,249
Schneeweiss, S., Rassen, J. A., Glynn, R. J., Avorn, J., Mogun, H., & Brookhart, M. A. (2009) · Epidemiology · doi:10.1097/ede.0b013e3181a663cc · Cited by 1,128
Self-citationJneid, H. M., Anderson, J. L., Wright, R. S., Adams, C. D., Bridges, C. R., Casey, D. E., Ettinger, S. M., Fesmire, F. M., Ganiats, T. G., Lincoff, A. M., et al. (2012) · Circulation · doi:10.1161/cir.0b013e318256f1e0 · Cited by 1,091
Rubin, D. B. (2007) · Statistics in Medicine · doi:10.1002/sim.2739 · Cited by 1,047
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Cepeda, M. S. (2003) · American Journal of Epidemiology · doi:10.1093/aje/kwg115 · Cited by 859
Analysis of Observational Studies in the Presence of Treatment Selection Bias
Stukel, T. A., Fisher, E. S., Wennberg, D. E., Alter, D. A., Gottlieb, D. J., & Vermeulen, M. J. (2007) · JAMA · doi:10.1001/jama.297.3.278 · Cited by 789
Systematic review of treatments for atopic eczema.
Hoare, Po, A. L. W., & Williams (2000) · Health Technology Assessment · doi:10.3310/hta4370 · Cited by 789
Little, R. J., & Rubin, D. B. (2000) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.21.1.121 · Cited by 721 · cited 1×
“Analyses using PS methods attempt to emulate randomized comparisons because they allow contrasts between patient groups that are on average similar on all observed confounders; however, PS methods cannot adjust for unmeasured confounding.(3,12,13) The literature on ACS offers a unique opportunity for assessing agreeme…”
MethodWhy do we need some large, simple randomized trials?
Yusuf, S., Collins, R., & Peto, R. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030421 · Cited by 700
Stürmer, T., Joshi, M., Glynn, R. J., Avorn, J., Rothman, K. J., & Schneeweiß, S. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.07.004 · Cited by 668
Holm, J., Nowicka, P., Farpour‐Lambert, N. J., O’Malley, G., Hassapidou, M. N., Weiss, R., & Baker, J. L. (2014) · Obesity Facts · doi:10.1159/000365773 · Cited by 596
Combining Propensity Score Matching with Additional Adjustments for Prognostic Covariates
Rubin, D. B., & Thomas, N. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474233 · Cited by 568
Randomized Trials or Observational Tribulations?
Pocock, S. J., & Elbourne, D. R. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422511 · Cited by 550
Combining Propensity Score Matching with Additional Adjustments for Prognostic Covariates
Thomas, N. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810725.025 · Cited by 508
Shah, B. R., Laupacis, A., Hux, J. E., & Austin, P. C. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.016 · Cited by 493
An overview of the objectives of and the approaches to propensity score analyses
Heinze, G., & Jüni, P. (2011) · European Heart Journal · doi:10.1093/eurheartj/ehr031 · Cited by 379
2011 Writing Group Members, Wright, R. S., Anderson, J. L., Adams, C. D., Bridges, C. R., Casey, D. E., Ettinger, S. M., Fesmire, F. M., Ganiats, T. G., Jneid, H. M., et al. (2011) · Circulation · doi:10.1161/cir.0b013e31820f2f3e · Cited by 359
MacLehose, Reeves, Harvey, Sheldon, Russell, & Black (2000) · Health Technology Assessment · doi:10.3310/hta4340 · Cited by 311
Austin, P. C., Grootendorst, P. V., Normand, S. T., & Anderson, G. M. (2006) · Statistics in Medicine · doi:10.1002/sim.2618 · Cited by 230
Evidence for Use of Coronary Stents: A Hierarchical Bayesian Meta-Analysis
Brophy, J., Belisle, P., & Joseph, L. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-10-200305200-00005 · Cited by 208
Why Data Bases Should Not Replace Randomized Clinical Trials
Byar, D. P. (1980) · Biometrics · doi:10.2307/2529989 · Cited by 206
Austin, P. C. (2008) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.108.790634 · Cited by 153
Observational Methods in Comparative Effectiveness Research
Concato, J., Lawler, E. V., Lew, R., Gaziano, J. M., Aslan, M., & Huang, G. D. (2010) · The American Journal of Medicine · doi:10.1016/j.amjmed.2010.10.004 · Cited by 129
Any casualties in the clash of randomised and observational evidence?
Ioannidis, J. P. A., Haidich, A., & Lau, J. (2001) · BMJ · doi:10.1136/bmj.322.7291.879 · Cited by 107
Rassen, J. A., Avorn, J., & Schneeweiß, S. (2010) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1867 · Cited by 49
The Washington Manual of medical therapeutics
Cooper, D., Krainik, A. J., Lubner, S. J., & Reno, H. (2007) · Lippincott Williams & Wilkins eBooks · Cited by 47
Ottervanger, J. P., Armstrong, P. W., Barnathan, E. S., Boersma, E. H., COOPER, J., Ohman, E. M., James, S. K., Wallentin, L. C., & Simoons, M. L. (2004) · European Heart Journal · doi:10.1016/j.ehj.2004.07.004 · Cited by 35
The Washington Manual of medical therapeutics
Green, G. B., & Schaiff, R. (2004) · Lippincott Williams & Wilkins eBooks · Cited by 31
Therapeutic Innovations, Diminishing Returns, and Control Rate Preservation
Kent, D. M. (2009) · JAMA · doi:10.1001/jama.2009.1679 · Cited by 30
Research works printed but not linked: 1
- 2002, Observational Studies
Cited by
Sorted by citations
Liddle, A. D., Judge, A., Pandit, H. G., & Murray, D. W. (2014) · The Lancet · doi:10.1016/s0140-6736(14)60419-0 · Cited by 676
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534
Wang, S. V., Schneeweiss, S., Initiative, R.-D., Franklin, J. M., Desai, R. J., Feldman, W., Garry, E. M., Glynn, R. J., Lin, K. J., Paik, J., et al. (2023) · JAMA · doi:10.1001/jama.2023.4221 · Cited by 428
When and how can real world data analyses substitute for randomized controlled trials?
Franklin, J. M., & Schneeweiss, S. (2017) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.857 · Cited by 316
Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies
Franklin, J. M., Patorno, E., Desai, R. J., Glynn, R. J., Martin, D., Quinto, K., Pawar, A. M., Bessette, L. G., Lee, H., Garry, E. M., et al. (2020) · Circulation · doi:10.1161/circulationaha.120.051718 · Cited by 316
Metadata
- Authors
- Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D.
- Affiliations
- Tufts Medical Center (US)
- Tufts University (US)
- Harvard University (US)
- Johns Hopkins University (US)
- Brigham and Women's Hospital (US)
- Lahey Hospital and Medical Center (US)
- Corresponding author
- Dahabreh, Issa J.
- ORCID
- Venue
- European Heart Journal
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-06-17
- Volume
- 33
- Issue
- 15
- Pages
- 1893-1901
- ISSN
- 0195-668X (ISSN-L)
- 1522-9645
- PMC
- PMC3409422
- OpenAlex
- W2162409127
- Cited by
- 204
- References
- 40
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 10 · 2023: 10 · 2022: 23 · 2021: 22 · 2020: 18 · 2019: 14 · 2018: 18 · 2017: 15 · 2016: 12 · 2015: 13 · 2014: 24 · 2013: 10 · 2012: 2
- FWCI
- 14.97
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- European Heart Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Center for Research Resources (UL1RR025752)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- propensity score methods, acute coronary syndrome, observational studies, randomized controlled trials, treatment effect estimation, confounding, mortality
- MeSH terms
- Humans, Regression Analysis, Risk Factors, Reproducibility of Results, Randomized Controlled Trials as Topic, Treatment Outcome, Acute Coronary Syndrome, Propensity Score
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
