Cited by 204Advanced Causal Inference TechniquesMeta-analysis and systematic reviewsStatistical Methods in Epidemiology

The DOI matches its Crossref record; 39 of the 40 references it cites were found in a registry or an index.

Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes

Open access

Dahabreh, Issa J., Sheldrick, Radley C., Paulus, Jessica K., Chung, Mei, Varvarigou, Vasileia, Jafri, Haseeb, Rassen, Jeremy A., Trikalinos, Thomas A., Kitsios, Georgios D.·2012·European Heart Journal

DOI 10.1093/eurheartj/ehs114PMC PMC3409422

Open this paper in CoChat — free account

Abstract

AIMS: Randomized controlled trials (RCTs) are the gold standard for assessing the efficacy of therapeutic interventions because randomization protects from biases inherent in observational studies. Propensity score (PS) methods, proposed as a potential solution to confounding of the treatment-outcome association, are widely used in observational studies of therapeutic interventions for acute coronary syndromes (ACS). We aimed to systematically assess agreement between observational studies using PS methods and RCTs on therapeutic interventions for ACS. METHODS AND RESULTS: We searched for observational studies of interventions for ACS that used PS methods to estimate treatment effects on short- or long-term mortality. Using a standardized algorithm, we matched observational studies to RCTs based on patients' characteristics, interventions, and outcomes ('topics'), and we compared estimates of treatment effect between the two designs. When multiple observational studies or RCTs were identified for the same topic, we performed a meta-analysis and used the summary relative risk for comparisons. We matched 21 observational studies investigating 17 distinct clinical topics to 63 RCTs (median = 3 RCTs per observational study) for short-term (7 topics) and long-term (10 topics) mortality. Estimates from PS analyses differed statistically significantly from randomized evidence in two instances; however, observational studies reported more extreme beneficial treatment effects compared with RCTs in 13 of 17 instances (P = 0.049). Sensitivity analyses limited to large RCTs, and using alternative meta-analysis models yielded similar results. CONCLUSION: For the treatment of ACS, observational studies using PS methods produce treatment effect estimates that are of more extreme magnitude compared with those from RCTs, although the differences are rarely statistically significant.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1093/eurheartj/ehs114

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Heart Disease and Stroke Statistics—2023 Update: A Report From the American Heart Association
Tsao, C. W., Aday, A. W., Almarzooq, Z. I., Anderson, C. A. M., Arora, P., Avery, C. L., Baker-Smith, C. M., Beaton, A. Z., Boehme, A. K., Buxton, A. E., et al. (2023) · Circulation · doi:10.1161/cir.0000000000001123 · Cited by 8,300
Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Heart Disease and Stroke Statistics—2011 Update: A Report From the American Heart Association
Roger, V. L., Go, A. S., Lloyd-Jones, D. M., Adams, R. J., Berry, J. D., Brown, T. M., Carnethon, M. R., Dai, S., de Simone, G., Ford, E. S., et al. (2011) · Circulation · doi:10.1161/cir.0b013e3182009701 · Cited by 6,149
ACC/AHA Guidelines for the Management of Patients With Unstable Angina and Non–ST-Segment Elevation Myocardial Infarction: Executive Summary and Recommendations
Braunwald, E., Antman, E. M., Beasley, J. W., Califf, R. M., Cheitlin, M. D., Hochman, J. S., Jones, R. H., Kereiakes, D. J., Kupersmith, J., Levin, T. N., et al. (2000) · Circulation · doi:10.1161/01.cir.102.10.1193 · Cited by 4,475
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
A comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
What is meant by intention to treat analysis? Survey of published randomised controlled trials
Hollis, S., & Campbell, F. (1999) · BMJ · doi:10.1136/bmj.319.7211.670 · Cited by 2,160
A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003
Austin, P. C. (2007) · Statistics in Medicine · doi:10.1002/sim.3150 · Cited by 1,249

Show all 40 references

Research works printed but not linked: 1

2002, Observational Studies

Sorted by citations

Adverse outcomes after total and unicompartmental knee replacement in 101 330 matched patients: a study of data from the National Joint Registry for England and Wales
Liddle, A. D., Judge, A., Pandit, H. G., & Murray, D. W. (2014) · The Lancet · doi:10.1016/s0140-6736(14)60419-0 · Cited by 676
Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534
Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials
Wang, S. V., Schneeweiss, S., Initiative, R.-D., Franklin, J. M., Desai, R. J., Feldman, W., Garry, E. M., Glynn, R. J., Lin, K. J., Paik, J., et al. (2023) · JAMA · doi:10.1001/jama.2023.4221 · Cited by 428
When and how can real world data analyses substitute for randomized controlled trials?
Franklin, J. M., & Schneeweiss, S. (2017) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.857 · Cited by 316
Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies
Franklin, J. M., Patorno, E., Desai, R. J., Glynn, R. J., Martin, D., Quinto, K., Pawar, A. M., Bessette, L. G., Lee, H., Garry, E. M., et al. (2020) · Circulation · doi:10.1161/circulationaha.120.051718 · Cited by 316
Safety and effectiveness of oral rivaroxaban versus standard anticoagulation for the treatment of symptomatic deep-vein thrombosis (XALIA): an international, prospective, non-interventional study
Ageno, W., Mantovani, L. G., Haas, S. K., Kreutz, R. H., Monje, D., Schneider, J., van Eickels, M., Gebel, M., Zell, E. R., & Turpie, A. G. G. (2015) · The Lancet Haematology · doi:10.1016/s2352-3026(15)00257-4 · Cited by 243
Comparison of Treatment Effect Estimates From Prospective Nonrandomized Studies With Propensity Score Analysis and Randomized Controlled Trials of Surgical Procedures
Lonjon, G., Boutron, I., Trinquart, L., Ahmad, N., Aïm, F., Nizard, R., & Ravaud, P. (2013) · Annals of Surgery · doi:10.1097/sla.0000000000000256 · Cited by 240
Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey
Hemkens, L. G., Contopoulos‐Ioannidis, D. G., & Ioannidis, J. P. A. (2016) · BMJ · doi:10.1136/bmj.i493 · Cited by 220
Review of the emerging evidence demonstrating the efficacy of ivermectin in the prophylaxis and treatment of COVID-19.
Kory, P., Meduri, G. U., Varon, J., Iglesias, J., & Marik, P. E. (2021) · American Journal of Therapeutics · doi:10.1097/mjt.0000000000001377 · Cited by 203
Adjusted Analyses in Studies Addressing Therapy and Harm
Agoritsas, T., Merglen, A., Shah, N. D., O'Donnell, M. J., & Guyatt, G. H. (2017) · JAMA · doi:10.1001/jama.2016.20029 · Cited by 186
Survival Advantage of Laparoscopic Versus Open Resection For Colorectal Liver Metastases
Syn, N. L. X., Kabir, T., Koh, Y. X., Tan, H. L., Wang, L. Z., Chin, B. Z., Wee, I. J. Y., Teo, J. Y., Tai, B. C., & Goh, B. K. P. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003672 · Cited by 145
Long-Term Health Outcomes in Women With Silicone Gel Breast Implants
Balk, E. M., Earley, A., Avendano, E. E., & Raman, G. (2015) · Annals of Internal Medicine · doi:10.7326/m15-1169 · Cited by 144
Long-term Outcomes of Laparoscopic Versus Open Surgery for Clinical Stage II/III Gastric Cancer
Kinoshita, T., Uyama, I., Terashima, M., Noshirο, H., Nagai, E., Obama, K., Tamamori, Y., Nabae, T., Honda, M., & Abe, T. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002768 · Cited by 143
The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal
Hettle, R., Corbett, M. S., Hinde, S., Hodgson, R. M., Jones-Diette, J. S., Woolacott, N. F., & Palmer, S. (2017) · Health Technology Assessment · doi:10.3310/hta21070 · Cited by 127
Nonrandomized real-world evidence to support regulatory decision making: process for a randomized trial replication project
Franklin, J. M., Pawar, A., Martin, D., Glynn, R. J., Levenson, M., Temple, R., & Schneeweiss, S. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1633 · Cited by 119
Use of Health Care Databases to Support Supplemental Indications of Approved Medications
Fralick, M., Kesselheim, A. S., Avorn, J., & Schneeweiß, S. (2017) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2017.3919 · Cited by 119
Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making
Sarri, G., Patorno, E., Yuan, H., Guo, J., Bennett, D., Wen, X., Zullo, A. R., Largent, J. A., Panaccio, M. P., Gokhale, M., et al. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111493 · Cited by 121
“Several reviews have found little difference between the evidence from observational studies and RCTs, 48–50 but counterexamples exist.”
Statins for Prevention of Cardiovascular Events in a Low-Risk Population With Low Ankle Brachial Index
Ramos, R., García-Gil, M., Comas‐Cufí, M., Quesada, M., Marrugat, J., Elosúa, R., Sala, J., Grau, M., Martí‐Lluch, R., Ponjoan, A., et al. (2016) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2015.11.052 · Cited by 117
“Researchers have demonstrated that such studies properly replicate the effects of statins observed in RCTs (42); it also has been suggested that observational studies could tend to overestimate the effect size of interventions (43) if several key points are not well addressed.”
Anticoagulant vs. antiplatelet therapy in patients with cryptogenic stroke and patent foramen ovale: an individual participant data meta-analysis
Kent, D. M., Dahabreh, I. J., Ruthazer, R., Furlan, A. J., Weimar, C., Serena, J. L., Meier, B., Mattle, H. P., Angelantonio, E. D., Paciaroni, M., et al. (2015) · European Heart Journal · doi:10.1093/eurheartj/ehv252 · Cited by 112
“sometimes disagree with randomized study results.(27,28) Fourth, the quality of study procedures can vary from database to database, with variable patient selection criteria, rates of loss to follow-up, and outcome ascertainment methods across databases.”
Background
Co-administration of furosemide with albumin for overcoming diuretic resistance in patients with hypoalbuminemia: A meta-analysis
Kitsios, G. D., Mascari, P., Ettunsi, R., & Gray, A. W. (2013) · Journal of Critical Care · doi:10.1016/j.jcrc.2013.10.004 · Cited by 105
“Our review did not consider observational studies that can inform on the pragmatic effectiveness of interventions [24].”
Background
Comparing Propensity Score Methods Versus Traditional Regression Analysis for the Evaluation of Observational Data: A Case Study Evaluating the Treatment of Gram-Negative Bloodstream Infections
Amoah, J., Stuart, E. A., Cosgrove, S. E., Harris, A. D., Han, J., Lautenbach, E., & Tamma, P. D. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa169 · Cited by 96
Association of Alendronate and Risk of Cardiovascular Events in Patients With Hip Fracture
Sing, C., Wong, A. Y. S., Kiel, D. P., Cheung, E. Y. N., Lam, J. K. Y., Cheung, T. T., Chan, E. W., Kung, A. W. C., Wong, I. C. K., & Cheung, C. (2018) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.3448 · Cited by 94
“Overestimation of treatment effect in PS-based observational studies is commonly reported.((31,32)) One possible reason for the discrepancy is the difference in the study populations.”
Background
Can We Trust Observational Studies Using Propensity Scores in the Critical Care Literature? A Systematic Comparison With Randomized Clinical Trials*
Kitsios, G. D., Dahabreh, I. J., Callahan, S. P., Paulus, J. K., Campagna, A. C., & Dargin, J. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001135 · Cited by 92
“Assessment of the Methodological Quality of PS Studies and RCTs Based on surveys of the methodological features of studies using PS methods (10, 13, 14), we identified a set of items as potentially indicative of the validity of PS-based analyses (supplemental document, Supplemental Digital Content 1, http://”
BackgroundMethodResultInfluential
Long-term Outcomes of Laparoscopic Versus Open Surgery for Clinical Stage I Gastric Cancer
Honda, M., Hiki, N., Kinoshita, T., Yabusaki, H., Abe, T., Nunobe, S., Terada, M., Matsuki, A., Sunagawa, H., Aizawa, M., et al. (2016) · Annals of Surgery · doi:10.1097/sla.0000000000001654 · Cited by 82
THE INTERNAL AND EXTERNAL VALIDITY OF THE REGRESSION DISCONTINUITY DESIGN: A META‐ANALYSIS OF 15 WITHIN‐STUDY COMPARISONS
Chaplin, D., Cook, T. D., Zurovac, J., Coopersmith, J., Finucane, M. M., Vollmer, L., & Morris, R. E. (2018) · Journal of Policy Analysis and Management · doi:10.1002/pam.22051 · Cited by 81
Comparative Effectiveness of Management Strategies for Renal Artery Stenosis: An Updated Systematic Review
Raman, G., Adam, G. P., Halladay, C. W., Langberg, V. N., Azodo, I. A., & Balk, E. M. (2016) · Annals of Internal Medicine · doi:10.7326/m16-1053 · Cited by 81
Machine learning methods for developing precision treatment rules with observational data
Kessler, R. C., Bossarte, R. M., Luedtke, A. R., Zaslavsky, A. M., & Zubizarreta, J. R. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103412 · Cited by 85
“It has been shown that analyses of balanced databases often yield aggregate results about comparative treatment effects very similar to those obtained in randomized clinical trials (Anglemyer, Horvath, & Bero, 2014; Dahabreh et al., 2012).”
Result
International multicentre propensity score-matched analysis comparing robotic versus laparoscopic right posterior sectionectomy
Chiow, A. K. H., Fuks, D., Choi, G. H., Syn, N. L. X., Sucandy, I., Marino, M. V., Prieto, M., Chong, C. C. N., Lee, J. H., Efanov, M. G., et al. (2021) · British journal of surgery · doi:10.1093/bjs/znab321 · Cited by 78
Psoriasis Increased the Risk of Adverse Cardiovascular Outcomes: A New Systematic Review and Meta-Analysis of Cohort Study
Liu, L., Cui, S., Liu, M., Huo, X., Zhang, G., & Wang, N. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.829709 · Cited by 76
“It has been shown that the practical meaning of HR and RR are very similar, so we converted HR directly to RR for merging and the final effect size is represented by RR (16).”
Method
Emulating a Novel Clinical Trial Using Existing Observational Data. Predicting Results of the PreVent Study
Admon, A. J., Donnelly, J. P., Casey, J. D., Janz, D. R., Russell, D. W., Joffe, A. M., Vonderhaar, D. J., Dischert, K. M., Stempek, S. B., Dargin, J., et al. (2019) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201903-241oc · Cited by 73
Can the Learning Health Care System Be Educated With Observational Data?
Dahabreh, I. J., & Kent, D. M. (2014) · JAMA · doi:10.1001/jama.2014.4364 · Cited by 71
The codex of science: honesty, precision, and truth--and its violations
Lüscher, T. F. (2013) · European Heart Journal · doi:10.1093/eurheartj/eht063 · Cited by 67
Opportunities and challenges in using studies without a control group in comparative effectiveness reviews
Paulus, J. K., Dahabreh, I. J., Balk, E. M., Avendano, E. E., Lau, J., & Ip, S. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1101 · Cited by 66
“Furthermore, empirical evidence indicates that in many cases, different study designs reach similar conclusions (Britton et al., 1998; Dahabreh et al., 2012; Dalziel et al., 2005; Ioannidis et al., 2001; MacLehose et al., 2000; Vandenbroucke, 2011), offering another rationale for quantitatively…”
Result
Tenets for the Proper Conduct and Use of Meta-Analyses: A Practical Guide for Neurosurgeons
Lee, K. S., Zhang, J. J. Y., Nga, V. D. W., Ng, C. H., Tai, B. C., Higgins, J. P. T., & Syn, N. L. X. (2022) · World Neurosurgery · doi:10.1016/j.wneu.2021.09.034 · Cited by 63
Low Serum Potassium Levels Increase the Infectious-Caused Mortality in Peritoneal Dialysis Patients: A Propensity-Matched Score Study
Ribeiro, S. C., Figueiredo, A. E., Barretti, P., Pecoits‐Filho, R., & de Moraes, T. P. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127453 · Cited by 63
“To overcome this limitation, we used a propensity score to mimic some of the characteristics of randomized clinical trials (RCT) instead of relying on the use of regression adjustments to account for differences at the baseline [13].”
Method
Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study
Toews, I., Anglemyer, A., Nyirenda, J. L., Alsaid, D., Balduzzi, S., Grummich, K., Schwingshackl, L., & Bero, L. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub3 · Cited by 60
Integration of Pain Score and Morphine Consumption in Analgesic Clinical Studies
Dai, F., Silverman, D. G., Chelly, J. E., Li, J., Belfer, I., & Li, Q. (2013) · Journal of Pain · doi:10.1016/j.jpain.2013.04.004 · Cited by 59
Intravenous Bone Marrow Mononuclear Cells for Acute Ischemic Stroke: Safety, Feasibility, and Effect Size from a Phase I Clinical Trial
Vahidy, F., Haque, M. E., Rahbar, M. H., Zhu, H., Rowan, P. J., Aisiku, I. P., Lee, D. A., Juneja, H. S., Alderman, S., Barreto, A. D., et al. (2019) · Stem Cells · doi:10.1002/stem.3080 · Cited by 53
Applications of propensity score methods in observational comparative effectiveness and safety research: where have we come and where should we go?
Borah, B. J., Moriarty, J. P., Crown, W. H., & Doshi, J. A. (2013) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.13.89 · Cited by 53
Supplemental dose of antithrombin use in disseminated intravascular coagulation patients after abdominal sepsis
Matsui, H., Fushimi, K., Yasunaga, H., & Tagami, T. (2015) · Thrombosis and Haemostasis · doi:10.1160/th15-01-0053 · Cited by 53
“However, when the study participants were restricted to patients with sepsis (40) and with coronary artery syndrome (41), the results were suggested conflicting one.”
Result
Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System
Kessler, R. C., Bauer, M. S., Bishop, T. M., Demler, O. V., Dobscha, S. K., Gildea, S. M., Goulet, J. L., Karras, E., Kreyenbuhl, J., Landes, S. J., et al. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00390 · Cited by 52
Risk of dementia or cognitive impairment in COPD patients: A meta-analysis of cohort studies
Wang, J., Li, X., Lei, S., Zhang, D., Zhang, S., Zhang, H., & Li, J. (2022) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2022.962562 · Cited by 51
“In the results section, all relative effects metrics were referred to as RRs, which did not affect our results or their interpretation (Dahabreh et al., 2012).”
Result
Review of the Emerging Evidence Demonstrating the Efficacy of Ivermectin in the Prophylaxis and Treatment of COVID-19Preprint
Kory, P. D., Meduri, G. U., Iglesias, J. I., Varón, J., Berkowitz, K., Kornfeld, H., Vinjevoll, E. H., Scott, M., Wagshul, F. A., & Marik, P. E. (2020) · doi:10.31219/osf.io/wx3zn · Cited by 49
Use of Open Claims vs Closed Claims in Health Outcomes Research
Başer, O., Samayoa, G., Yapar, N., Baser, E., & Mete, F. (2023) · Journal of health economics and outcomes research · doi:10.36469/001c.87538 · Cited by 48
Methodological choices for the clinical development of medical devices
Bernard, A., Vaneau, M., Fournel, I., Galmiche, H., Nony, P., & Dubernard, J. M. (2014) · Medical Devices Evidence and Research · doi:10.2147/mder.s63869 · Cited by 47
“A systematic review that compared the results of RCTs with those of observational studies using propensity score on the various treatments during acute coronary syndromes has been published.(59) Overall, observational studies report a greater amount of effect for long-term mortality than RCTs.”
Limited liver resections in the posterosuperior segments: international multicentre propensity score-matched and coarsened exact-matched analysis comparing the laparoscopic and robotic approaches
D’Silva, M., Han, H., Liu, R., Kingham, T. P., Choi, G. H., Syn, N. L. X., Prieto, M., Choi, S. H., Sucandy, I., Chiow, A. K. H., et al. (2022) · British journal of surgery · doi:10.1093/bjs/znac270 · Cited by 46
Benchmarking Observational Analyses Against Randomized Trials: a Review of Studies Assessing Propensity Score Methods
Forbes, S. P., & Dahabreh, I. (2020) · Journal of general internal medicine · doi:10.1007/s11606-020-05713-5 · Cited by 35
What type of clinical evidence is needed to assess medical devices?
Tarricone, R., Boscolo, P. R., & Armeni, P. (2016) · European Respiratory Review · doi:10.1183/16000617.0016-2016 · Cited by 43
Outcomes Associated With Inferior Vena Cava Filters Among Patients With Thromboembolic Recurrence During Anticoagulant Therapy
Mellado, M., Pijoán, J. I., Jiménez, D., Muriel, A., Aujesky, D., Bertoletti, L., Decousus, H., Barrios, D., Clarà, A. B. I., Yusen, R. D., et al. (2016) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2016.08.039 · Cited by 42
“Studies have found good agreement between the treatment effects in the propensity score studies compared with those in the randomized trials (23).”
Result
Development and Validation of a Machine Learning Individualized Treatment Rule in First-Episode Schizophrenia
Wu, C., Luedtke, A. R., Sadikova, E., Tsai, H., Liao, S., Liu, C., Gau, S. S., VanderWeele, T. J., & Kessler, R. C. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.21660 · Cited by 46
Observational studies using propensity score analysis underestimated the effect sizes in critical care medicine
Zhang, Z., Ni, H., & Xu, X. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.02.018 · Cited by 51
Using Trial and Observational Data to Assess Effectiveness: Trial Emulation, Transportability, Benchmarking, and Joint Analysis
Dahabreh, I. J., Matthews, A., Steingrimsson, J. A., Scharfstein, D. O., & Stuart, E. A. (2023) · Epidemiologic Reviews · doi:10.1093/epirev/mxac011 · Cited by 40
Performing both propensity score and instrumental variable analyses in observational studies often leads to discrepant results: a systematic review
Laborde‐Castérot, H., Agrinier, N., & Thilly, N. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.003 · Cited by 38
Comparing treatment effects between propensity scores and randomized controlled trials: improving conduct and reporting
Collins, P. G. S., & Manach, Y. L. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs186 · Cited by 37
“observational studies using propensity scores and RCTs evaluating therapeutic interventions for acute coronary syndrome.(5) By”
Method
Assessment of the Association of Bilateral Internal Thoracic Artery Skeletonization and Sternal Wound Infection After Coronary Artery Bypass Grafting
Rubens, F. D., Chen, L., & Bourke, M. (2015) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2015.10.031 · Cited by 32
Methodological challenges for the evaluation of clinical effectiveness in the context of accelerated regulatory approval: an overview
Woolacott, N. F., Corbett, M. S., Jones-Diette, J. S., & Hodgson, R. M. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.07.002 · Cited by 32
A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments
Cuello‐García, C. A., Morgan, R. L., Brożek, J., Santesso, N. A., Verbeek, J. H. A. M., Thayer, K. A., Guyatt, G. H., & Schünemann, H. J. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.010 · Cited by 33
Using observational data for personalized medicine when clinical trial evidence is limited
Gershman, B., Guo, D. P., & Dahabreh, I. J. (2018) · Fertility and Sterility · doi:10.1016/j.fertnstert.2018.04.005 · Cited by 33
The relationship between cancer patients' place of death and bereaved caregivers' mental health status
Hatano, Y., Aoyama, M., Morita, T., Yamaguchi, T., Maeda, I., Kizawa, Y., Tsuneto, S., Shima, Y., & Miyashita, M. (2017) · Psycho-Oncology · doi:10.1002/pon.4412 · Cited by 32
“Recent systematic reviews show that the estimates of treatment effects from nonrandomized controlled studies with propensity score analysis were consistent with the results of randomized controlled trials.(24,25) Our results would suggest that the effect of the place of death on bereavement outcomes may be more modest…”
Result
Associations of Bariatric Interventions With Micronutrient and Endocrine Disturbances
Syn, N. L. X., Lee, P. C., Kovalik, J., Tham, K. W., Ong, H. S., Chan, W. H., Tan, C. S., & Lim, C. H. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.5123 · Cited by 30
Reduced Incidence and Better Liver Disease Outcomes among Chronic HCV Infected Patients Who Consume Cannabis
Adejumo, A., Adegbala, O. M., Adejumo, K. L., & Bukong, T. N. (2018) · Canadian Journal of Gastroenterology and Hepatology · doi:10.1155/2018/9430953 · Cited by 29
External control arm analysis: an evaluation of propensity score approaches, G-computation, and doubly debiased machine learning
Loiseau, N., Trichelair, P., He, M., Andreux, M., Zaslavskiy, M., Wainrib, G., & Blum, M. G. B. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01799-z · Cited by 29
XALIA: rationale and design of a non-interventional study of rivaroxaban compared with standard therapy for initial and long-term anticoagulation in deep vein thrombosis
Ageno, W., Mantovani, L. G., Haas, S. K., Kreutz, R. H., Haupt, V., Schneider, J., & Turpie, A. G. (2014) · Thrombosis Journal · doi:10.1186/1477-9560-12-16 · Cited by 27
“The potential for allocation bias in open-label, nonrandomized studies will be minimized by also presenting a propensity score analysis to control the effect of unbalanced covariates [26,27].”
Clinical data mining: challenges, opportunities, and recommendations for translational applications
Qiao, H., Chen, Y., Qian, C., & Guo, Y. (2024) · Journal of Translational Medicine · doi:10.1186/s12967-024-05005-0 · Cited by 32
“Data mining of EHRs has been demonstrated to be able to replicate the findings of clinical trials [118–120], and to be more realistically assessed than in RCTs given their size and multifariousness of patients.”
Background
First experience without pre-ballooning in transapical aortic valve implantation: a propensity score-matched analysis†
Kempfert, J., Meyer, A., Kim, W.-K., Linden, A. V., Arsalan, M., Blumenstein, J., Möllmann, H., & Walther, T. (2014) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezu097 · Cited by 26
“In addition, PS matching has been shown to be a reliable statistical method [18] but in general is inferior to a prospective randomized trial as PS matching cannot adjust for any potential unknown/unmeasured confounder [19].”
Method
Suboptimal Use of Inpatient Palliative Care Consultation May Lead to Higher Readmissions and Costs in End-Stage Liver Disease
Adejumo, A., Kim, D., Iqbal, U., Yoo, E. R., Boursiquot, B. C., Cholankeril, G. T., Wong, R. J., Kwo, P. Y., & Ahmed, A. (2019) · Journal of Palliative Medicine · doi:10.1089/jpm.2019.0100 · Cited by 26
“Propensity matching approximates a pseudorandomized clinical study and produces estimates similar in size to double-blinded clinical studies.(26,27) After matching, we compared between PCC and noPCC using the statistical method for paired comparison such as paired t test, Wilcoxon signed-rank test, and McNemar’s test.”
Method
Translating cell therapies for neurodegenerative diseases: Huntington’s disease as a model disorder
Rosser, A., Busse, M. E., Gray, W., Badin, R. A., Perrier, A. L., Wheelock, V., Cozzi, E., Perpiñá, U., Salado-Manzano, C., Mills, L. J., et al. (2022) · Brain · doi:10.1093/brain/awac086 · Cited by 29
“the use of historical control data or observational data from disease registries, that methods to account for confounders are also taken into account.(57,58) Given the very early stage of development of these novel therapeutics in a rare disease such asHuntington’s disease, itmay be important to start by focusing on s…”
The effect of labor epidural analgesia on labor, delivery, and neonatal outcomes: a propensity score-matched analysis in a single Japanese institute
Naito, Y., Ida, M., Yamamoto, R., Tachibana, K., & Kinouchi, K. (2019) · JA Clinical Reports · doi:10.1186/s40981-019-0260-z · Cited by 26
Efficacy and Safety of iGlarLixi, Fixed-Ratio Combination of Insulin Glargine and Lixisenatide, Compared with Basal-Bolus Regimen in Patients with Type 2 Diabetes: Propensity Score Matched Analysis
Tabák, Á. G., Anderson, J. E., Aschner, P. J., Liu, M., Saremi, A., Stella, P., Tinahones, F. J., Wysham, C., & Meier, J. J. (2019) · Diabetes Therapy · doi:10.1007/s13300-019-00735-7 · Cited by 29
Current challenges for clinical trials of cardiovascular medical devices
Zannad, F. M., Stough, W. G., Piña, I. L., Mehran, R., Abraham, W. T., Anker, S. D., Ferrari, G. M. D., Farb, A. A., Geller, N. L., Kieval, R. S., et al. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.05.021 · Cited by 24
“For example, clinical outcomes in subjects enrolled in observational studies may appear to be better than those seen in randomized trials [30], or they may differ completely [31].”
Background
Cost-effectiveness of a mindfulness-based mental health promotion program: economic evaluation of a nonrandomized controlled trial with propensity score matching
Müller, G., Pfinder, M., Schmahl, C. G., Bohus, M., & Lyssenko, L. (2019) · BMC Public Health · doi:10.1186/s12889-019-7585-4 · Cited by 27
Comparison of treatment effect estimates of non-vitamin K antagonist oral anticoagulants versus warfarin between observational studies using propensity score methods and randomized controlled trials
Li, G., Holbrook, A. M., Jin, Y., Zhang, Y., Levine, M., Mbuagbaw, L. C. E., Witt, D. M., Crowther, M. A., Connolly, S. J., Chai‐Adisaksopha, C., et al. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0178-y · Cited by 26
“Several systematic reviews have compared the treatment effect estimates from PS studies and RCTs in different clinical settings, with a conclusion that PS studies may either overestimate [24, 26], underestimate [27], or be consistent with [25, 28, 29] the estimates from RCTs.”
Result
Assessing Heterogeneity of Treatment Effects in Observational Studies
Robertson, S. E., Leith, A., Schmid, C. H., & Dahabreh, I. J. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa235 · Cited by 23
Impact of the matching algorithm on the treatment effect estimate: A neutral comparison study
Heinz, P., Wendel‐Garcia, P. D., & Held, U. (2022) · Biometrical Journal · doi:10.1002/bimj.202100292 · Cited by 25
“Results based on observational data may reflect clinical reality more closely, but, on the other hand, bias and confounding are common to affect treatment effect estimates, producing potentially conflicting evidence as compared to RCT results (Dahabreh et al., 2012).”
Result
Comparing surgical strategies for end-stage anteromedial osteoarthritis
Mikkelsen, M., Wilson, H. A., Gromov, K., Price, A. J., & Troelsen, A. (2022) · Bone & Joint Open · doi:10.1302/2633-1462.35.bjo-2021-0174.r1 · Cited by 23
Head-to-Head Comparison of Sirolimus-Eluting Stents versus Paclitaxel-Eluting Stents in Patients Undergoing Percutaneous Coronary Intervention: A Meta-Analysis of 76 Studies
Zhang, X., Xie, J., Li, G., Chen, Q., & Xu, B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0097934 · Cited by 23
“In a comprehensive meta-analysis comparing safety and efficiency of DES with BMS, DES significantly reduces mortality by 22% in observational studies, but the effect is attenuated in RCTs (RR = 0.97, 95% CI = 0.81–1.15) [21].”
BackgroundInfluential
Why the Propensity for Propensity Scores?*
Day, A. G. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001175 · Cited by 21
“In 2012, Dahabreh et al (5) concluded that PS studies of acute coronary syndromes tended to overestimate the efficacy of experimental treatments compared with RCTs.”
Result
Blumgart Anastomosis After Pancreaticoduodenectomy. A Comprehensive Systematic Review, Meta‐Analysis, and Meta‐Regression
Ricci, C., Ingaldi, C., Alberici, L., Pagano, N., Mosconi, C., Marasco, G., Minni, F., & Casadei, R. (2021) · World Journal of Surgery · doi:10.1007/s00268-021-06039-x · Cited by 22
Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials
Hong, Y. D., Jansen, J. P., Guerino, J., Berger, M. L., Crown, W. H., Goettsch, W., Mullins, C. D., Willke, R. J., & Orsini, L. S. (2021) · BMC Medicine · doi:10.1186/s12916-021-02176-1 · Cited by 22
“43) [16] and absence of an extreme difference.”
Background
Comparison between short and long‐term outcomes after minimally invasive versus open primary liver resections for hepatocellular carcinoma: A 1:1 matched analysis
Goh, B. K. P., Syn, N. L. X., Koh, Y. X., Teo, J., Cheow, P. C., Jeyaraj, P. R., Chow, P. K. H., Ooi, L. L., Chung, A. Y. F., & Chan, C. Y. (2021) · Journal of Surgical Oncology · doi:10.1002/jso.26556 · Cited by 22
Effects of prescribed antihypertensives and other cardiovascular drugs on mortality in patients with atrial fibrillation and hypertension: a cohort study from Sweden
Carlsson, A. C., Wändell, P. E., Sundquist, K., Johansson, S., & Sundquist, J. (2014) · Hypertension Research · doi:10.1038/hr.2014.32 · Cited by 23
Sex differences in the outcome after percutaneous coronary intervention – A propensity matching analysis
Ndrepepa, G., Kufner, S., Mayer, K., Cassese, S., Xhepa, E., Fusaro, M., Hasimi, E., Schüpke, S., Laugwitz, K., Schunkert, H., et al. (2018) · Cardiovascular revascularization medicine · doi:10.1016/j.carrev.2018.05.010 · Cited by 22
Randomized clinical trials of cardiovascular disease in obstructive sleep apnea: understanding and overcoming bias
McEvoy, R. D., Sánchez‐de‐la‐Torre, M., Peker, Y., Anderson, C. S., Redline, S. S., & Barbé, F. (2021) · SLEEP · doi:10.1093/sleep/zsab019 · Cited by 22
“Propensity score and other adjusted analyses provide information on the utility of evidence in real-world settings but cannot establish effectiveness, may overestimate CV treatment effects, and even conflict with RCT findings [19].”
Background
Comparison of 10-year clinical wear of annealed and remelted highly cross-linked polyethylene: A propensity-matched cohort study
Hamai, S., Nakashima, Y., Mashima, N., Yamamoto, T., Kamada, T., Motomura, G., Imai, H., Fukushi, J., Miura, H., & Iwamoto, Y. (2015) · Journal of the mechanical behavior of biomedical materials/Journal of mechanical behavior of biomedical materials · doi:10.1016/j.jmbbm.2015.12.022 · Cited by 19
Reappraisal of Non-vitamin K Antagonist Oral Anticoagulants in Atrial Fibrillation Patients: A Systematic Review and Meta-Analysis
Liu, F., Yang, Y., Cheng, W., Ma, J., & Zhu, W. (2021) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.757188 · Cited by 18
“An effective method to evaluate interventions’ effectiveness in typical clinical settings can be provided by the propensity score (PS) (14).”
Method
Associations of Antidepressants With Atrial Fibrillation and Ventricular Arrhythmias: A Systematic Review and Meta-Analysis
Cao, Y., Zhou, M., Guo, H., & Zhu, W. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.840452 · Cited by 18
“Ultimately, we selected RRs and 95% CIs to represent all the effect estimates in the results, which did not influence the final interpretation (14).”
Method
Use of real world data to improve drug coverage decisions in China
Wang, W., Tan, J., Wu, J., Huang, S., Huang, Y., Xie, F., & Sun, X. (2023) · BMJ · doi:10.1136/bmj-2021-068911 · Cited by 18
The past, present and perhaps future of pharmacovigilance: homage to Folke Sjoqvist
Moore, N. D. (2013) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-013-1486-8 · Cited by 18
Can propensity score matching be applied to cross-sectional data to evaluate Community-Based Rehabilitation? Results of a survey implementing the WHO’s Community-Based Rehabilitation indicators in Vietnam
Mason, C., Sabariego, C., Thắng, Đ. M., & Weber, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-022544 · Cited by 17
“A systematic review comparing 21 PSM studies to 63 randomised controlled studies (RCTs) on therapeutic interventions for acute coronary syndromes found that PSM produced more extreme treatment effect estimates when compared with those from RCTs, although these differences were rarely statistically significant.(13) A s…”
Result
Total hip arthroplasty after periacetabular osteotomy versus primary total hip arthroplasty: a propensity-matched cohort study
Komiyama, K., Hamai, S., Motomura, G., Ikemura, S., Fujii, M., Kawahara, S., & Nakashima, Y. (2021) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-021-03817-z · Cited by 16
Effectiveness of influenza vaccination for children in Japan: Four-year observational study using a large-scale claims database
Shibata, N., Kimura, S. H., Hoshino, T., Takeuchi, M., & Urushihara, H. (2018) · Vaccine · doi:10.1016/j.vaccine.2018.03.082 · Cited by 16
Misoprostol as an adjunct to oxytocin can reduce postpartum-haemorrhage: a propensity score–matched retrospective chart review in Bamenda-Cameroon, 2015–2016
Morfaw, F., Fundoh, M., Pisoh, C. T., Ayaba, B., Mbuagbaw, L. C. E., Anderson, L. N., & Thabane, L. (2019) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-019-2407-3 · Cited by 16
Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions
Mei, F., Yao, M., Wang, Y., Huan, J., Ma, Y., Li, G., Zou, K., Li, L., & Sun, X. (2024) · BMC Medicine · doi:10.1186/s12916-024-03778-1 · Cited by 15
“RoRs indicated an “important difference” (< 0.70 or > 1.43) or not (0.7 ≤ RoR ≤ 1.43) [27, 28].”
Influential
Addressing Multimorbidity in Evidence Integration and Synthesis
Trikalinos, T. A., Segal, J. B., & Boyd, C. M. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-013-2661-4 · Cited by 16
Statins and new-onset atrial fibrillation in a cohort of patients with hypertension. Analysis of electronic health records, 2006–2015
Alves‐Cabratosa, L., García-Gil, M., Comas‐Cufí, M., Ponjoan, A., Martí‐Lluch, R., Parramon, D., Blanch, J., Elosua-Bayés, M., & Ramos, R. (2017) · PLoS ONE · doi:10.1371/journal.pone.0186972 · Cited by 15
Liver Resection Versus Embolization for Recurrent Hepatocellular Carcinoma
Midorikawa, Y., Takayama, T., Moriguchi, M., Yagi, R., Yamagishi, S., Nakayama, H., Aramaki, O., Yamazaki, S., Tsuji, S., & Higaki, T. (2019) · World Journal of Surgery · doi:10.1007/s00268-019-05225-2 · Cited by 15
Effectiveness of a Comprehensive Mental Health Literacy Educational Programme for Junior High School Students: A Randomised Controlled Trial Examining Changes in Their Knowledge, Attitudes, and Behaviour
Mori, R., Uchino, T., Mizuno, M., Yamaguchi, T., Katagiri, N., & Nemoto, T. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12081281 · Cited by 28
“For example, using methods such as propensity score matching may be useful [43,44].”
Method
Minimally invasive versus open right anterior sectionectomy and central hepatectomy for central liver malignancies: a propensity‐score‐matched analysis
Chin, K. M., Linn, Y. L., Cheong, C. K., Koh, Y. X., Teo, J., Chung, A. Y. F., Chan, C. Y., & Goh, B. K. P. (2021) · ANZ Journal of Surgery · doi:10.1111/ans.16719 · Cited by 15
“PSM is thus deemed to be a close approximation to randomization and is one of the most commonly employed methods for evaluation of differing treatment modalities in the current surgical literature.(47,48) While most previous studies have performed PSM based on pre-operative parameters of qualitative hepatic function (…”
Method
Laparoscopic versus open liver resection for hepatocellular carcinoma in elderly patients: A systematic review and meta-analysis of propensity score-matched studies
Wang, S., Ye, G., Wang, J., Xu, S., Ye, Q., & Ye, H. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.939877 · Cited by 14
A palliative care approach for adult non-cancer patients with life-limiting illnesses is cost-saving or cost-neutral: a systematic review of RCTs
Janke, K., Salifu, Y., Gavini, S., Preston, N. J., & Gadoud, A. (2024) · BMC Palliative Care · doi:10.1186/s12904-024-01516-1 · Cited by 14
“Observational methods can overestimate beneficial treatment effects compared to RCTs [30], which might explain our more nuanced findings for RCTs.”

Authors: Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D.
Affiliations: Tufts Medical Center (US)
Tufts University (US)
Harvard University (US)
Johns Hopkins University (US)
Brigham and Women's Hospital (US)
Lahey Hospital and Medical Center (US)
Corresponding author: Dahabreh, Issa J.
ORCID: Dahabreh, Issa J.
Sheldrick, Radley C.
Paulus, Jessica K.
Chung, Mei
Rassen, Jeremy A.
Trikalinos, Thomas A.
Kitsios, Georgios D.
Venue: European Heart Journal
Publisher: Oxford University Press (OUP)
Published: 2012-06-17
Volume: 33
Issue: 15
Pages: 1893-1901
ISSN: 0195-668X (ISSN-L)
1522-9645
DOI: 10.1093/eurheartj/ehs114
PMC: PMC3409422
OpenAlex: W2162409127
Cited by: 204
References: 40
Citations by year: 2026: 4 · 2025: 9 · 2024: 10 · 2023: 10 · 2022: 23 · 2021: 22 · 2020: 18 · 2019: 14 · 2018: 18 · 2017: 15 · 2016: 12 · 2015: 13 · 2014: 24 · 2013: 10 · 2012: 2
FWCI: 14.97
Type: journal article
Language: en
Open access: green
europepmc.org
Locations: European Heart Journal (publishedVersion)
Europe PMC (PubMed Central) (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
Funders: National Center for Research Resources (UL1RR025752)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: propensity score methods, acute coronary syndrome, observational studies, randomized controlled trials, treatment effect estimation, confounding, mortality
MeSH terms: Humans, Regression Analysis, Risk Factors, Reproducibility of Results, Randomized Controlled Trials as Topic, Treatment Outcome, Acute Coronary Syndrome, Propensity Score

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks