Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study

Toews, Ingrid; Anglemyer, Andrew; Nyirenda, John LZ; Alsaid, Dima; Balduzzi, Sara; Grummich, Kathrin; Schwingshackl, Lukas; Bero, Lisa

doi:10.1002/14651858.mr000034.pub3

Cited by 60Meta-analysis and systematic reviewsAdvanced Causal Inference TechniquesStatistical Methods in Epidemiology

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study

Open access

Toews, Ingrid, Anglemyer, Andrew, Nyirenda, John LZ, Alsaid, Dima, Balduzzi, Sara, Grummich, Kathrin, Schwingshackl, Lukas, Bero, Lisa·2024·Cochrane Database of Systematic Reviews

DOI 10.1002/14651858.mr000034.pub3PMC PMC10765475

Open this paper in CoChat — free account Open PDF

Abstract

BACKGROUND: Researchers and decision-makers often use evidence from randomised controlled trials (RCTs) to determine the efficacy or effectiveness of a treatment or intervention. Studies with observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and their modifications (including both randomised and observational designs) are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population. An up-to-date systematic analysis is needed to identify differences in effect estimates from RCTs and observational studies. This updated review summarises the results of methodological reviews that compared the effect estimates of observational studies with RCTs from evidence syntheses that addressed the same health research question. OBJECTIVES: To assess and compare synthesised effect estimates by study type, contrasting RCTs with observational studies. To explore factors that might explain differences in synthesised effect estimates from RCTs versus observational studies (e.g. heterogeneity, type of observational study design, type of intervention, and use of propensity score adjustment). To identify gaps in the existing research comparing effect estimates across different study types. SEARCH METHODS: We searched MEDLINE, the Cochrane Database of Systematic Reviews, Web of Science databases, and Epistemonikos to May 2022. We checked references, conducted citation searches, and contacted review authors to identify additional reviews. SELECTION CRITERIA: We included systematic methodological reviews that compared quantitative effect estimates measuring the efficacy or effectiveness of interventions tested in RCTs versus in observational studies. The included reviews compared RCTs to observational studies (including retrospective and prospective cohort, case-control and cross-sectional designs). Reviews were not eligible if they compared RCTs with studies that had used some form of concurrent allocation. DATA COLLECTION AND ANALYSIS: Using results from observational studies as the reference group, we examined the relative summary effect estimates (risk ratios (RRs), odds ratios (ORs), hazard ratios (HRs), mean differences (MDs), and standardised mean differences (SMDs)) to evaluate whether there was a relatively larger or smaller effect in the ratio of odds ratios (ROR) or ratio of risk ratios (RRR), ratio of hazard ratios (RHR), and difference in (standardised) mean differences (D(S)MD). If an included review did not provide an estimate comparing results from RCTs with observational studies, we generated one by pooling the estimates for observational studies and RCTs, respectively. Across all reviews, we synthesised these ratios to produce a pooled ratio of ratios comparing effect estimates from RCTs with those from observational studies. In overviews of reviews, we estimated the ROR or RRR for each overview using observational studies as the reference category. We appraised the risk of bias in the included reviews (using nine criteria in total). To receive an overall low risk of bias rating, an included review needed: explicit criteria for study selection, a complete sample of studies, and to have controlled for study methodological differences and study heterogeneity. We assessed reviews/overviews not meeting these four criteria as having an overall high risk of bias. We assessed the certainty of the evidence, consisting of multiple evidence syntheses, with the GRADE approach. MAIN RESULTS: ≥ 50% (ratio of ratios 1.11, 95% CI 1.04 to 1.18); - no use (ratio of ratios 1.07, 95% CI 1.03 to 1.11) or unclear use (ratio of ratios 1.13, 95% CI 1.03 to 1.25) of propensity score adjustment in observational studies; and - observational studies without further specification of the study design (ratio of ratios 1.06, 95% CI 0.96 to 1.18). We detected no clear difference in other subgroup analyses. AUTHORS' CONCLUSIONS: We found no difference or a very small difference between effect estimates from RCTs and observational studies. These findings are largely consistent with findings from recently published research. Factors other than study design need to be considered when exploring reasons for a lack of agreement between results of RCTs and observational studies, such as differences in the population, intervention, comparator, and outcomes investigated in the respective studies. Our results underscore that it is important for review authors to consider not only study design, but the level of heterogeneity in meta-analyses of RCTs or observational studies. A better understanding is needed of how these factors might yield estimates reflective of true effectiveness.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1002/14651858.mr000034.pub3

Cites

168 of 168 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cisgender men and women have fewer risk factors for late-life dementia on average than transgender men or women.
“However, cisgender men and women have fewer risk factors for late-life dementia on average than transgender men or women. (63)”
From Dementia prevention, intervention, and care: 2024 report of the Lancet standing Commission (2024) · cites this paper as [63]
The Discussion’s Summary of main results compares effect estimates from randomized and observational studies. Dementia appears in Data availability only as an example of a possible outcome; the passages found by searching for transgender, cisgender, and dementia do not compare dementia risk factors by gender identity. Our copy is truncated, so the claim cannot be verified from it. (LLM, conf=0.74)
Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline
Campbell, M., McKenzie, J. E., Sowden, A., Katikireddi, S. V., Brennan, S. E., Ellis, S., Hartmann-Boyce, J., Ryan, R., Shepperd, S., Thomas, J., et al. (2020) · BMJ · doi:10.1136/bmj.l6890 · Cited by 4,628 · see citing works
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563 · see citing works
Interaction revisited: the difference between two estimates.
Altman, D. G., & Bland, J. M. (2003) · BMJ · doi:10.1136/bmj.326.7382.219 · Cited by 3,203 · see citing works
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907 · see citing works
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884 · see citing works
A comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347 · see citing works
The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · see citing works
Bias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model
Viechtbauer, W. (2005) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986030003261 · Cited by 1,699 · see citing works
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465 · see citing works
Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials
Theodoratou, E., Tzoulaki, I., Zgaga, L., & Ioannidis, J. P. A. (2014) · BMJ · doi:10.1136/bmj.g2035 · Cited by 956 · see citing works
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919 · see citing works
Effects of regular aspirin on long-term cancer incidence and metastasis: a systematic comparison of evidence from observational studies versus randomised trials
Algra, A. M., & Rothwell, P. M. (2012) · The Lancet Oncology · doi:10.1016/s1470-2045(12)70112-2 · Cited by 833 · see citing works
Effect of aspirin on long-term risk of colorectal cancer: consistent evidence from randomised and observational studies
Flossmann, E., & Rothwell, P. M. (2007) · The Lancet · doi:10.1016/s0140-6736(07)60747-8 · Cited by 820 · see citing works
Analysis of Observational Studies in the Presence of Treatment Selection Bias
Stukel, T. A., Fisher, E. S., Wennberg, D. E., Alter, D. A., Gottlieb, D. J., & Vermeulen, M. J. (2007) · JAMA · doi:10.1001/jama.297.3.278 · Cited by 789 · see citing works
The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials
Kunz, R., & Oxman, A. D. (1998) · BMJ · doi:10.1136/bmj.317.7167.1185 · Cited by 770 · see citing works
Alternative approaches for confounding adjustment in observational studies using weighting based on the propensity score: a primer for practitioners
Desai, R. J., & Franklin, J. M. (2019) · BMJ · doi:10.1136/bmj.l5657 · Cited by 657 · see citing works
The Ethics of Childhood Obesity Treatment - from the Childhood Obesity Task Force (COTF) of European Association for the Study of Obesity (EASO)
Holm, J., Nowicka, P., Farpour‐Lambert, N. J., O’Malley, G., Hassapidou, M. N., Weiss, R., & Baker, J. L. (2014) · Obesity Facts · doi:10.1159/000365773 · Cited by 596 · see citing works
Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534 · see citing works
Self-citation
Guidelines for reporting meta-epidemiological methodology research
Murad, M. H., & Wang, Z. (2017) · Evidence Based Medicine · doi:10.1136/ebmed-2017-110713 · Cited by 515 · see citing works
Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review
Shah, B. R., Laupacis, A., Hux, J. E., & Austin, P. C. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.016 · Cited by 493 · see citing works
Randomisation to protect against selection bias in healthcare trials
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000012.pub3 · Cited by 439 · see citing works
Methods in health services research: Interpreting the evidence: choosing between randomised and non-randomised studies
McKee, M., Britton, A. R., Black, N. A., McPherson, K., Sanderson, C. F. B., & Bain, C. (1999) · BMJ · doi:10.1136/bmj.319.7205.312 · Cited by 396 · see citing works
A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies
Bosco, J. L. F., Silliman, R. A., Thwin, S. S., Geiger, A. M., Buist, D. S. M., Prout, M. N., Yood, M. U., Haque, R., Wei, F., & Lash, T. L. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.03.001 · Cited by 394 · see citing works
Long-acting injectable versus oral antipsychotics for the maintenance treatment of schizophrenia: a systematic review and comparative meta-analysis of randomised, cohort, and pre-post studies
Kishimoto, T., Hagi, K., Kurokawa, S., Kane, J. M., & Correll, C. U. (2021) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00039-0 · Cited by 369 · see citing works
Safety and efficacy of digoxin: systematic review and meta-analysis of observational and controlled trial data
Ziff, O. J., Lane, D. A., Samra, M., Griffith, M., Kirchhof, P., Lip, G. Y. H., Steeds, R. P., Townend, J., & Kotecha, D. (2015) · BMJ · doi:10.1136/bmj.h4451 · Cited by 341 · see citing works
An extended mixed‐effects framework for meta‐analysis
Sera, F., Armstrong, B., Blangiardo, M., & Gasparrini, A. (2019) · Statistics in Medicine · doi:10.1002/sim.8362 · Cited by 334 · see citing works
Choosing between randomised and non-randomised studies: a systematic review.
Britton, McKee, M., Black, McPherson, Sanderson, C. F. B., & Bain (1998) · Health Technology Assessment · doi:10.3310/hta2130 · Cited by 330 · see citing works
A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies
MacLehose, Reeves, Harvey, Sheldon, Russell, & Black (2000) · Health Technology Assessment · doi:10.3310/hta4340 · Cited by 311 · see citing works
Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies
Franklin, J. M., Patorno, E., Desai, R. J., Glynn, R. J., Martin, D., Quinto, K., Pawar, A. M., Bessette, L. G., Lee, H., Garry, E. M., et al. (2020) · Circulation · doi:10.1161/circulationaha.120.051718 · Cited by 301 · see citing works
Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview
Golder, S., Loke, Y. K., & Bland, M. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1001026 · Cited by 270 · see citing works
Risk of fractures with inhaled corticosteroids in COPD: systematic review and meta-analysis of randomised controlled trials and observational studies
Loke, Y. K., Cavallazzi, R., & Singh, S. (2011) · Thorax · doi:10.1136/thx.2011.160028 · Cited by 268 · see citing works
The role of method in treatment effectiveness research: Evidence from meta-analysis.
Wilson, D. B., & Lipsey, M. W. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.413 · Cited by 261 · see citing works
Comparing effects of kyphoplasty, vertebroplasty, and non-surgical management in a systematic review of randomized and non-randomized controlled studies
Papanastassiou, I. D., Phillips, F. M., Meirhaeghe, J. V., Berenson, J. R., Andersson, G. B. J., Chung, G., Small, B. J., Aghayev, K. V., & Vrionis, F. D. (2012) · European Spine Journal · doi:10.1007/s00586-012-2314-z · Cited by 243 · see citing works
Systematic Review: Benefits and Harms of In-Hospital Use of Recombinant Factor VIIa for Off-Label Indications
Yank, V. A., Tuohy, C. V., Logan, A. C., Bravata, D. M., Staudenmayer, K., Eisenhut, R., H, V. S. M. P., McMahon, D., Olkin, I., McDonald, K. M., et al. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-154-8-201104190-00004 · Cited by 240 · see citing works
Comparison of Treatment Effect Estimates From Prospective Nonrandomized Studies With Propensity Score Analysis and Randomized Controlled Trials of Surgical Procedures
Lonjon, G., Boutron, I., Trinquart, L., Ahmad, N., Aïm, F., Nizard, R., & Ravaud, P. (2013) · Annals of Surgery · doi:10.1097/sla.0000000000000256 · Cited by 236 · see citing works
Statin Therapy and Outcome After Ischemic Stroke
Chróinín, D. N., Asplund, K., Åsberg, S., Callaly, E. L., Cuadrado‐Godia, E., Díez‐Tejedor, E., Napoli, M. D., Engelter, S. T., Furie, K. L., Giannopoulos, S., et al. (2013) · Stroke · doi:10.1161/strokeaha.112.668277 · Cited by 235 · see citing works
Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies
Papanikolaou, P. N. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050873 · Cited by 229 · see citing works
Worldwide Collaborative Observational Study and Meta‐Analysis on Allogenic Leukocyte Immunotherapy for Recurrent Spontaneous Abortion1
The Recurrent Miscarriage Immunotherapy Trialists Group, Coulam, C. B., Clark, D. A., Collins, J., Scott, J. R., Schlesselman, J., Aoki, K., Carp, H., Cauchi, M. N., Lim, D., et al. (1994) · American Journal of Reproductive Immunology · doi:10.1111/j.1600-0897.1994.tb01095.x · Cited by 221 · see citing works
Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey
Hemkens, L. G., Contopoulos‐Ioannidis, D. G., & Ioannidis, J. P. A. (2016) · BMJ · doi:10.1136/bmj.i493 · Cited by 220 · see citing works
Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials
Abraham, N. S., Byrne, C. J., Young, J. M., & Solomon, M. J. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.04.005 · Cited by 209 · see citing works
Do observational studies using propensity score methods agree with randomized trials? a systematic comparison of studies on acute coronary syndromes
Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs114 · Cited by 204 · see citing works
Operative versus nonoperative treatment of proximal humeral fractures: a systematic review, meta-analysis, and comparison of observational studies and randomized controlled trials
Beks, R. B., Ochen, Y., Frima, H., Smeeing, D. P. J., van der Meijden, O. A. J., Timmers, T. K., van der Velde, D., van Heijl, M., Leenen, L. P. H., Groenwold, R. H. H., et al. (2018) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2018.03.009 · Cited by 198 · see citing works
Leisure-Time Physical Activity in Pregnancy and Maternal-Child Health: A Systematic Review and Meta-Analysis of Randomized Controlled Trials and Cohort Studies
da Silva, S. G., Ricardo, L. I. C., Evenson, K. R., & Hallal, P. C. (2016) · Sports Medicine · doi:10.1007/s40279-016-0565-2 · Cited by 192 · see citing works
Comparison of predictions based on observational data with the results of randomized controlled clinical trials of coronary artery bypass surgery
Hlatky, M. A., Califf, R. M., Harrell, F. E., Lee, K. L., Mark, D. B., & Pryor, D. B. (1988) · Journal of the American College of Cardiology · doi:10.1016/0735-1097(88)90086-1 · Cited by 180 · see citing works
Frequency of Persistent Tooth Pain after Root Canal Therapy: A Systematic Review and Meta-Analysis
Nixdorf, D. R., Moana-Filho, E. J., Law, A., McGuire, L., Hodges, J. S., & John, M. T. (2010) · Journal of Endodontics · doi:10.1016/j.joen.2009.11.007 · Cited by 179 · see citing works
GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines
Cuello‐García, C. A., Santesso, N. A., Morgan, R. L., Verbeek, J. H. A. M., Thayer, K. A., Ansari, M., Meerpohl, J. J., Schwingshackl, L., Katikireddi, S. V., Brożek, J., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.026 · Cited by 178 · see citing works
Self-citation
Efficacy and Effectiveness of Depot Versus Oral Antipsychotics in Schizophrenia
Kirson, N. Y., Weiden, P. J., Yermakov, S., Huang, W., Samuelson, T., Offord, S. J., Greenberg, P. E., & Wong, B. J. O. (2013) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.12r08167 · Cited by 177 · see citing works
Comparison of Population-Based Observational Studies With Randomized Trials in Oncology
Soni, P., Hartman, H. E., Dess, R. T., Abugharib, A. E., Allen, S. G., Feng, F. Y.-C., Zietman, A. L., Jagsi, R., Schipper, M. J., & Spratt, D. E. (2019) · Journal of Clinical Oncology · doi:10.1200/jco.18.01074 · Cited by 172 · see citing works
An easy mathematical proof showed that time-dependent bias inevitably leads to biased effect estimation
Beyersmann, J., Gastmeier, P., Wolkewitz, M., & Schumacher, M. C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.02.008 · Cited by 165 · see citing works
Off-Pump Coronary Artery Bypass Surgery and Acute Kidney Injury: A Meta-analysis of Randomized and Observational Studies
Nigwekar, S. U., Kandula, P., Hix, J. K., & Thakar, C. V. (2009) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2009.01.267 · Cited by 154 · see citing works
Survival Advantage of Laparoscopic Versus Open Resection For Colorectal Liver Metastases
Syn, N. L. X., Kabir, T., Koh, Y. X., Tan, H. L., Wang, L. Z., Chin, B. Z., Wee, I. J. Y., Teo, J. Y., Tai, B. C., & Goh, B. K. P. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003672 · Cited by 145 · see citing works
Use of observational databases to evaluate the effectiveness of antiretroviral therapy for HIV infection: comparison of cohort studies with randomized trials
Phillips, A. N., Grabar, S., Tassie, J.-M., Costagliola, D. G., Lundgren, J. D., & Egger, M. (1999) · AIDS · doi:10.1097/00002030-199910220-00010 · Cited by 136 · see citing works
Surgical Versus Nonsurgical Treatment for Midshaft Clavicle Fractures in Patients Aged 16 Years and Older: A Systematic Review, Meta-analysis, and Comparison of Randomized Controlled Trials and Observational Studies
Smeeing, D. P. J., van der Ven, D. J. C., Hietbrink, F., Timmers, T. K., van Heijl, M., Kruyt, M. C., Groenwold, R. H. H., van der Meijden, O. A. J., & Houwert, R. M. (2016) · The American Journal of Sports Medicine · doi:10.1177/0363546516673615 · Cited by 127 · see citing works
A review and meta-analysis of hormonal treatment of cryptorchidism.
Pyörälä, S., Huttunen, N., & Uhari, M. K. (1995) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jcem.80.9.7673426 · Cited by 125 · see citing works
Conflicting Results Between Randomized Trials and Observational Studies on the Impact of Proton Pump Inhibitors on Cardiovascular Events When Coadministered With Dual Antiplatelet Therapy
Melloni, C., Washam, J. B., Jones, W. S., Halim, S. A., Hasselblad, V., Mayer, S., Heidenfelder, B. L., & Dolor, R. J. (2015) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.114.001177 · Cited by 122 · see citing works
An introduction to methodological issues when including non‐randomised studies in systematic reviews on the effects of interventions
Reeves, B. C., Higgins, J. P. T., Ramsay, C. R., Shea, B., Tugwell, P., & Wells, G. A. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1068 · Cited by 111 · see citing works
Basic study design influences the results of orthodontic clinical investigations
Papageorgiou, S. N., Xavier, G. M., & Cobourne, M. T. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.008 · Cited by 108 · see citing works
A meta-analysis of the therapeutic role of oil soluble contrast media at hysterosalpingography: a surprising result?
Watson, A., Vandekerckhove, P., Lilford, R. J., Vail, A., Brosens, I. A., & Hughes, E. G. (1994) · Fertility and Sterility · doi:10.1016/s0015-0282(16)56578-9 · Cited by 107 · see citing works
Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery
Kuß, O., Legler, T., & Börgermann, J. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.005 · Cited by 99 · see citing works
Risk of Major Gastrointestinal Bleeding With New vs Conventional Oral Anticoagulants: A Systematic Review and Meta-analysis
Gu, Z., Wei, A., Zhang, C., Wang, X., Zhang, L., Shen, L., Li, Z., Pan, M.-M., Liu, X.-Y., Pu, J., et al. (2019) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2019.05.056 · Cited by 98 · see citing works
Breast Cancer Risk and Duration of Estrogen Use
STEINBERG, K. K., Smith, S. J., Thacker, S. B., & Stroup, D. F. (1994) · Epidemiology · doi:10.1097/00001648-199407000-00007 · Cited by 98 · see citing works
Randomized trials versus observational studies in adolescent pregnancy prevention
Guyatt, G. H. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00160-2 · Cited by 94 · see citing works
Non-steroidal anti-inflammatory drugs and myocardial infarctions: comparative systematic review of evidence from observational studies and randomised controlled trials
Scott, P. A., Kingsley, G. H., Smith, C. M., Choy, E., & Scott, D. L. (2007) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2006.068650 · Cited by 94 · see citing works
Can We Trust Observational Studies Using Propensity Scores in the Critical Care Literature? A Systematic Comparison With Randomized Clinical Trials*
Kitsios, G. D., Dahabreh, I. J., Callahan, S. P., Paulus, J. K., Campagna, A. C., & Dargin, J. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001135 · Cited by 92 · see citing works
Comparison of Effects in Randomized Controlled Trials With Observational Studies in Digestive Surgery
Shikata, S., Nakayama, T., Noguchi, Y., Taji, Y., & Yamagishi, H. (2006) · Annals of Surgery · doi:10.1097/01.sla.0000225356.04304.bc · Cited by 91 · see citing works
The clinical course of low back pain: a meta-analysis comparing outcomes in randomised clinical trials (RCTs) and observational studies
Artus, M., van der Windt, D., Jordan, K. P., & Croft, P. R. (2014) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-15-68 · Cited by 90 · see citing works
Retention in opioid agonist treatment: a rapid review and meta-analysis comparing observational studies and randomized controlled trials
Klimas, J., Hamilton, M.-A., Gorfinkel, L., Adam, A., Cullen, W., & Wood, E. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01764-9 · Cited by 90 · see citing works
Using Multiple Types of Studies in Systematic Reviews of Health Care Interventions – A Systematic Review
Peinemann, F., Tushabe, D. A., & Kleijnen, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0085035 · Cited by 89 · see citing works
Conservative vs. operative treatment for humeral shaft fractures: a meta-analysis and systematic review of randomized clinical trials and observational studies
van de Wall, B. J. M., Ochen, Y., Beeres, F. J. P., Babst, R. H., Link, B. C., Heng, M. M., van der Velde, D., Knobe, M., Groenwold, R. H. H., & Houwert, R. M. (2020) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2020.01.072 · Cited by 88 · see citing works
Prognostic effect size of cardiovascular biomarkers in datasets from observational studies versus randomised trials: meta-epidemiology study
Tzoulaki, I., Siontis, K. C., & Ioannidis, J. P. A. (2011) · BMJ · doi:10.1136/bmj.d6829 · Cited by 79 · see citing works
Hip fracture risk in relation to vitamin D supplementation and serum 25-hydroxyvitamin D levels: a systematic review and meta-analysis of randomised controlled trials and observational studies
Lai, J. K. C., Lucas, R. M., Clements, M., Roddam, A., & Banks, E. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-331 · Cited by 79 · see citing works
Primary endocrine therapy as a treatment for older women with operable breast cancer – A comparison of randomised controlled trial and cohort study findings
Morgan, J. L., Reed, M., & Wyld, L. (2014) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2014.02.224 · Cited by 78 · see citing works
Randomised controlled trials for policy interventions: a review of reviews and meta-regression
Oliver, S. R., Bagnall, A., Thomas, J., Shepherd, J., Sowden, A. J., White, I. R., Dinnes, J., Rees, R. W., Colquitt, J. L., Oliver, K. A., et al. (2010) · Health Technology Assessment · doi:10.3310/hta14160 · Cited by 76 · see citing works
Empirical Comparison of the Results of Randomized Controlled Trials and Case-Control Studies in Evaluating the Effectiveness of Screening Mammography
Demissie, K., Mills, O. F., & Rhoads, G. G. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00243-6 · Cited by 76 · see citing works
Bisphosphonates and atrial fibrillation: Bayesian meta-analyses of randomized controlled trials and observational studies
Mak, A., Cheung, M. W. ‐., Ho, R., Cheak, A. A.-C., & Lau, C. S. (2009) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-10-113 · Cited by 74 · see citing works
Rheumatoid arthritis patients treated in trial and real world settings: comparison of randomized trials with registries
Kilcher, G., Hummel, N., Didden, E., Egger, M., Reichenbach, S., & 4, F. T. G. W. P. (2017) · Rheumatology · doi:10.1093/rheumatology/kex394 · Cited by 71 · see citing works
Adding non-randomised studies to a Cochrane review brings complementary information for healthcare stakeholders: an augmented systematic review and meta-analysis
Arditi, C., Burnand, B., & Peytremann‐Bridevaux, I. (2016) · BMC Health Services Research · doi:10.1186/s12913-016-1816-5 · Cited by 68 · see citing works
A review and meta-analysis of hormonal treatment of cryptorchidism
Pyorala, S. (1995) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.80.9.2795 · Cited by 67 · see citing works
Antidepressant Response in Major Depressive Disorder: A Meta-Regression Comparison of Randomized Controlled Trials and Observational Studies
Naudet, F., Maria, A. S., & Falissard, B. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020811 · Cited by 66 · see citing works
Effectiveness of Long-term Psychodynamic Psychotherapy: A Meta-analysis
Leichsenring, F. (2008) · JAMA · doi:10.1001/jama.300.13.1551 · Cited by 65 · see citing works
A Model for Incorporating Historical Controls into a Meta-Analysis
Begg, C. B., & Pilote, L. (1991) · Biometrics · doi:10.2307/2532647 · Cited by 64 · see citing works
Evaluating agreement between bodies of evidence from randomised controlled trials and cohort studies in nutrition research: meta-epidemiological study
Schwingshackl, L., Balduzzi, S., Beyerbach, J., Bröckelmann, N., Werner, S. S., Zähringer, J., Nagavci, B., & Meerpohl, J. J. (2021) · BMJ · doi:10.1136/bmj.n1864 · Cited by 63 · see citing works
Self-citation
A comparison of randomized concurrent control groups with matched historical control groups: are historical controls valid?
Diehl, L., & Perry, D. J. (1986) · Journal of Clinical Oncology · doi:10.1200/jco.1986.4.7.1114 · Cited by 61 · see citing works
Hierarchy of evidence: differences in results between non-randomized studies and randomized trials in patients with femoral neck fractures
Bhandari, M., Tornetta, P., Ellis, T. J., Audigé, L., Sprague, S., Kuo, J. C., & Swiontkowski, M. F. (2003) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-003-0559-z · Cited by 55 · see citing works
Evidence from Nonrandomized Studies: A Case Study on the Estimation of Causal Effects
Schmoor, C., Caputo, A., & Schumacher, M. C. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn010 · Cited by 53 · see citing works
Bisphosphonates and osteoporotic fractures: a cross-design synthesis of results among compliant/persistent postmenopausal women in clinical practice versus randomized controlled trials
Wilkes, M. M., Navickis, R. J., Chan, W. W., & Lewiecki, E. M. (2009) · Osteoporosis International · doi:10.1007/s00198-009-0991-1 · Cited by 48 · see citing works
Comparison of efficacy of treatments for early syphilis: A systematic review and network meta-analysis of randomized controlled trials and observational studies
Liu, H., Han, Y., Chen, X., Li, B., Guo, S., Li, L., Wu, P., & Yin, Y.-P. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180001 · Cited by 47 · see citing works
Analysis of Observational Studies: A Guide to Understanding Statistical Methods
Morshed, S., Tornetta, P., & Bhandari, M. (2009) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.h.01577 · Cited by 43 · see citing works
Impact of random assignment on study outcome: An empirical examination
Ottenbacher, K. J. (1992) · Controlled Clinical Trials · doi:10.1016/0197-2456(92)90029-y · Cited by 41 · see citing works
Choice of observational study design impacts on measurement of antipsychotic risks in the elderly: a systematic review
Pratt, N. L., Roughead, E. E., Salter, A. B., & Ryan, P. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-72 · Cited by 41 · see citing works
Observational studies using propensity score analysis underestimated the effect sizes in critical care medicine
Zhang, Z., Ni, H., & Xu, X. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.02.018 · Cited by 41 · see citing works
A comparison of statistical methods for evaluating risk factor changes in community-based studies: An example from the Stanford Three-Community Study
Williams, P. T., Fortmann, S. P., Farquhar, J. W., Varady, A. N., & Mellen, S. (1981) · Journal of Chronic Diseases · doi:10.1016/0021-9681(81)90019-9 · Cited by 41 · see citing works
Updated systematic review and meta-analysis of the impact of contact force sensing on the safety and efficacy of atrial fibrillation ablation: discrepancy between observational studies and randomized control trial data
Virk, S. A., Ariyaratnam, J., Bennett, R. G., & Kumar, S. (2018) · EP Europace · doi:10.1093/europace/euy266 · Cited by 40 · see citing works
Validity of observational evidence on putative risk and protective factors: appraisal of 3744 meta-analyses on 57 topics
Janiaud, P., Agarwal, A., Tzoulaki, I., Τheodoratou, E., Tsilidis, K. K., Εvangelou, Ε., & Ioannidis, J. P. A. (2021) · BMC Medicine · doi:10.1186/s12916-021-02020-6 · Cited by 39 · see citing works
Efficacy of vildagliptin versus sulfonylureas as add-on therapy to metformin: comparison of results from randomised controlled and observational studies
Åhrén, B., Mathieu, C., Bader, G., Schweizer, A., & Foley, J. E. (2014) · Diabetologia · doi:10.1007/s00125-014-3222-z · Cited by 39 · see citing works
The selection of cases for randomised trials: a registry survey of concurrent trial and non-trial patients
for the British Stomach Cancer Group*, Ward, L., Fielding, J., Dunn, J., & Kelly, K. (1992) · British Journal of Cancer · doi:10.1038/bjc.1992.390 · Cited by 39 · see citing works
Methodological quality and homogeneity influenced agreement between randomized trials and nonrandomized studies of the same intervention for back pain
Furlan, A. D., TOMLINSON, G. A., Jadad, A. R., & Bombardier, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.04.019 · Cited by 38 · see citing works
Comparing treatment effects between propensity scores and randomized controlled trials: improving conduct and reporting
Collins, P. G. S., & Manach, Y. L. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs186 · Cited by 37 · see citing works
Conservative Management of Patent Ductus Arteriosus in Preterm Infants—A Systematic Review and Meta-Analyses Assessing Differences in Outcome Measures Between Randomized Controlled Trials and Cohort Studies
Hundscheid, T., Jansen, E. J. S., Onland, W., Kooi, E. M. W., Andriessen, P., & de Boode, W. P. (2021) · Frontiers in Pediatrics · doi:10.3389/fped.2021.626261 · Cited by 37 · see citing works
A New Method for Synthesizing Radiation Dose–Response Data From Multiple Trials Applied to Prostate Cancer
Díez, P., Vogelius, I. R., & Bentzen, S. M. (2009) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2009.06.013 · Cited by 36 · see citing works
Examining Heterogeneity in Meta-Analysis
Furlan, A. D., TOMLINSON, G. A., Jadad, A. R., & Bombardier, C. (2008) · Spine · doi:10.1097/brs.0b013e31816233b5 · Cited by 36 · see citing works
Effect of renin-angiotensin system inhibitors on mortality in heart failure with preserved ejection fraction: a meta-analysis of observational cohort and randomized controlled studies
Fukuta, H., Goto, T., Wakami, K., & Ohte, N. (2017) · Heart Failure Reviews · doi:10.1007/s10741-017-9637-0 · Cited by 34 · see citing works
A Propensity Score Matched Comparison of Clinical Outcomes in Atrial Fibrillation Patients Taking Vitamin K Antagonists: Comparing the “Real-World” vs Clinical Trials
Rivera‐Caravaca, J. M., Esteve‐Pastor, M. A., Marín, F. R., Valdés, M., Vicente, V., Roldán, V., & Lip, G. Y. H. (2018) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2018.01.028 · Cited by 33 · see citing works
The importance of adjusting for potential confounders in Bayesian hierarchical models synthesising evidence from randomised and non-randomised studies: an application comparing treatments for abdominal aortic aneurysms
McCarron, C. E., Pullenayegum, E., Thabane, L., Goeree, R. A., & Tarride, J. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-64 · Cited by 33 · see citing works
Cement augmentation for trochanteric femur fractures: A meta-analysis of randomized clinical trials and observational studies
Rompen, I. F., Knobe, M., Link, B.-C., Beeres, F. J. P., Baumgaertner, R., Diwersi, N., Migliorini, F., Nebelung, S., Babst, R. H., & van de Wall, B. J. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251894 · Cited by 33 · see citing works
Catheter ablation versus medical therapy for treatment of ventricular tachycardia associated with structural heart disease: Systematic review and meta-analysis of randomized controlled trials and comparison with observational studies
Anderson, R. D., Ariyarathna, N., Lee, G., Virk, S. A., Trivic, I., Campbell, T. G., Chow, C. K., Kalman, J. M., & Kumar, S. (2019) · Heart Rhythm · doi:10.1016/j.hrthm.2019.05.026 · Cited by 31 · see citing works
Response Rates and Durations of Response for Biomarker-Based Cancer Drugs in Nonrandomized Versus Randomized Trials
Gyawali, B., D’Andrea, E., Franklin, J. M., & Kesselheim, A. S. (2020) · Journal of the National Comprehensive Cancer Network · doi:10.6004/jnccn.2019.7345 · Cited by 31 · see citing works
Meta-Analysis of Efficacy and Safety of Proton Pump Inhibitors with Dual Antiplatelet Therapy for Coronary Artery Disease
Khan, S. U., Lone, A. N., Asad, Z. U. A., Rahman, H., Khan, M. S., Saleem, M. A., Arshad, A., Nawaz, N., Sattur, S., & Kaluski, E. (2019) · Cardiovascular revascularization medicine · doi:10.1016/j.carrev.2019.02.002 · Cited by 30 · see citing works
A systematic literature review on the efficacy–effectiveness gap: comparison of randomized controlled trials and observational studies of glucose-lowering drugs
Ankarfeldt, M. Z., Adalsteinsson, E., Groenwold, R. H. H., Ali, M. S., & Klungel, O. H. (2017) · Clinical Epidemiology · doi:10.2147/clep.s121991 · Cited by 29 · see citing works
Manic switches induced by antidepressants: an umbrella review comparing randomized controlled trials and observational studies
Allain, N., Leven, C., Falissard, B., Allain, J., Batail, J.-M. V., Polard, É., Montastruc, F., Drapier, D., & Naudet, F. (2016) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12672 · Cited by 29 · see citing works
Evaluating agreement between bodies of evidence from randomized controlled trials and cohort studies in medical research: a meta-epidemiological study
Bröckelmann, N., Balduzzi, S., Harms, L., Beyerbach, J., Πετροπούλου, Μ., Kubiak, C., Wolkewitz, M., Meerpohl, J. J., & Schwingshackl, L. (2022) · BMC Medicine · doi:10.1186/s12916-022-02369-2 · Cited by 29 · see citing works
Self-citation
Worldwide collaborative observational study and meta-analysis on allogenic leukocyte immunotherapy for recurrent spontaneous abortion. Recurrent Miscarriage Immunotherapy Trialists Group.
(1994) · PubMed · Cited by 25 · see citing works
Randomized versus nonrandomized studies of alcohol treatment: participants, methodological features and posttreatment functioning.
Moyer, A. E., & Finney, J. W. (2002) · Journal of Studies on Alcohol · doi:10.15288/jsa.2002.63.542 · Cited by 24 · see citing works
Comparison of treatment effect estimates of non-vitamin K antagonist oral anticoagulants versus warfarin between observational studies using propensity score methods and randomized controlled trials
Li, G., Holbrook, A. M., Jin, Y., Zhang, Y., Levine, M., Mbuagbaw, L. C. E., Witt, D. M., Crowther, M. A., Connolly, S. J., Chai‐Adisaksopha, C., et al. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0178-y · Cited by 24 · see citing works
Comparative Effectiveness of Initial Antiretroviral Therapy Regimens
Mugavero, M. J., May, M., Ribaudo, H. J., Gulick, R. M., Riddler, S. A., Haubrich, R., Napravnik, S., Abgrall, S., Phillips, A. N., Harris, R., et al. (2011) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0b013e318230372e · Cited by 23 · see citing works
Meta-analysis of coronary computed tomography angiography versus standard of care strategy for the evaluation of low risk chest pain: Are randomized controlled trials and cohort studies showing the same evidence?
El‐Hayek, G. E., Benjo, A. M., Uretsky, S., Al-Mallah, M. H., Cohen, R. E., Bamira, D. G., Chavez, P., Nascimento, F. O., Santana, O., Patel, R. A. G., et al. (2014) · International Journal of Cardiology · doi:10.1016/j.ijcard.2014.09.012 · Cited by 23 · see citing works
Agreement of treatment effects from observational studies and randomized controlled trials evaluating hydroxychloroquine, lopinavir-ritonavir, or dexamethasone for covid-19: meta-epidemiological study
Moneer, O., Daly, G., Skydel, J. J., Nyhan, K., Lurie, P. G., Ross, J. S., & Wallach, J. D. (2022) · BMJ · doi:10.1136/bmj-2021-069400 · Cited by 23 · see citing works
Differences in interaction and subgroup-specific effects were observed between randomized and nonrandomized studies in three empirical examples
Schmidt, A. F., Rovers, M. M., Klungel, O. H., Hoes, A. W., Knol, M. J., Nielen, M., de Boer, A., & Groenwold, R. H. H. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.08.008 · Cited by 22 · see citing works
Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials
Hong, Y. D., Jansen, J. P., Guerino, J., Berger, M. L., Crown, W. H., Goettsch, W., Mullins, C. D., Willke, R. J., & Orsini, L. S. (2021) · BMC Medicine · doi:10.1186/s12916-021-02176-1 · Cited by 22 · see citing works
Trials on the Effect of Cardiac Resynchronization on Arterial Blood Pressure in Patients With Heart Failure
Ather, S., Bangalore, S., Vemuri, S., Cao, L., Bozkurt, B., & Messerli, F. H. (2010) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2010.10.014 · Cited by 21 · see citing works
Randomized trial vs. observational study of acupuncture for migraine found that patient characteristics differed but outcomes were similar
Linde, K., Streng, A., Hoppe, A., Weidenhammer, W., Wagenpfeil, S., & Melchart, D. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.06.014 · Cited by 21 · see citing works
Meta-analytic comparison of randomized and nonrandomized studies of breast cancer surgery
Edwards, J. P., Kelly, E. W., Lin, Y., Lenders, T. J., Ghali, W. A., & Graham, A. P. (2012) · Canadian Journal of Surgery · doi:10.1503/cjs.023410 · Cited by 21 · see citing works
Do the observational studies using propensity score analysis agree with randomized controlled trials in the area of sepsis?
Zhang, Z., Ni, H., & Xu, X. (2014) · Journal of Critical Care · doi:10.1016/j.jcrc.2014.05.023 · Cited by 20 · see citing works
Comparison between treatment effects in a randomised controlled trial and an observational study using propensity scores in primary care
Stuart, B. L., Grebel, L. E., Butler, C., Hood, K., Verheij, T. J., & Little, P. S. (2017) · British Journal of General Practice · doi:10.3399/bjgp17x692153 · Cited by 20 · see citing works
Nonrandomized studies using causal-modeling may give different answers than RCTs: a meta-epidemiological study
Ewald, H., Ioannidis, J. P. A., Ladanie, A., Cord, K. A. M., Bucher, H. C., & Hemkens, L. G. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.10.012 · Cited by 20 · see citing works
A Bias in the Evaluation of Bias Comparing Randomized Trials with Nonexperimental Studies
Franklin, J. M., Dejene, S. Z., Huybrechts, K. F., Wang, S. V., Kulldorff, M., & Rothman, K. J. (2017) · Epidemiologic Methods · doi:10.1515/em-2016-0018 · Cited by 19 · see citing works
Evaluating Concordance of Bodies of Evidence from Randomized Controlled Trials, Dietary Intake, and Biomarkers of Intake in Cohort Studies: A Meta-Epidemiological Study
Beyerbach, J., Stadelmaier, J., HOFFMANN, G. E., Balduzzi, S., Bröckelmann, N., & Schwingshackl, L. (2021) · Advances in Nutrition · doi:10.1093/advances/nmab095 · Cited by 19 · see citing works
Self-citation
Concordance of randomized and nonrandomized studies was unrelated to translational patterns of two nutrient-disease associations
Trikalinos, T. A., Moorthy, D., Chung, M. C., Yu, W. W., Lee, J., Lichtenstein, A. H., & Lau, J. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.07.006 · Cited by 18 · see citing works
Direct Oral Anticoagulants vs. Vitamin-K Antagonists in the Elderly With Atrial Fibrillation: A Systematic Review Comparing Benefits and Harms Between Observational Studies and Randomized Controlled Trials
Shen, N., Wu, Y., Wang, N., Kong, L.-C., Zhang, C., Wang, J., Gu, Z., & Chen, J. (2020) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2020.00132 · Cited by 18 · see citing works
Systematic differences in effect estimates between observational studies and randomized control trials in meta-analyses in nephrology
Kimachi, M., Ōnishi, A., Tajika, A., Kimachi, K., & Furukawa, T. A. (2021) · Scientific Reports · doi:10.1038/s41598-021-85519-5 · Cited by 17 · see citing works
Comparison of randomized and non-randomized controlled trials evidence regarding the effectiveness of workplace exercise on musculoskeletal pain control
Moreira, R., Foltran, F. A., Alburquerque‐Sendín, F., Mancini, M. C., & Coury, H. J. C. G. (2012) · Work · doi:10.3233/wor-2012-0764-4782 · Cited by 17 · see citing works
No inexplicable disagreements between real-world data–based nonrandomized controlled studies and randomized controlled trials were found
Mathes, T., Rombey, T., Kuß, O., & Pieper, D. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.019 · Cited by 16 · see citing works
Agreement between study designs: a systematic review comparing observational studies and randomized trials of surgical treatments for necrotizing enterocolitis
van Heesewijk, A. E., Rush, M. L., Schmidt, B. K., Kirpalani, H. M., & DeMauro, S. B. (2018) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.1080/14767058.2018.1533948 · Cited by 15 · see citing works
Does Quality of Observational Studies Affect the Results of a Meta-Analysis?
Torre, G. L., de Waure, C., Specchia, M. L., Nicolotti, N., Capizzi, S., Bilotta, A., Clemente, G., & Ricciardi, W. (2009) · Pancreas · doi:10.1097/mpa.0b013e318190d795 · Cited by 14 · see citing works
Graduated compression stocking thromboprophylaxis for elderly inpatients
Labarère, J., Bosson, J., Sevestre, M., Delmas, A.-S., Dupas, S., Thenault, M.-H., Legagneux, A., Böge, G., Terriat, B., & Pernod, G. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00623.x · Cited by 14 · see citing works
Compared to randomized studies, observational studies may overestimate the effectiveness of DOACs: a metaepidemiological approach
Safieddine, M., Chapelle, C., Ollier, É., Ferdynus, C., Bertoletti, L., Mismetti, P., Cucherat, M., & Laporte, S. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.10.013 · Cited by 13 · see citing works
Expanding the Evidence Base
Atar, D., Ong, S., & Lansberg, P. J. (2012) · American Journal of Therapeutics · doi:10.1097/mjt.0b013e318245ce94 · Cited by 12 · see citing works
Reported effects in randomized controlled trials were compared with those of nonrandomized trials in cholecystectomy
Müeller, D., Sauerland, S., Neugebauer, E. A. M., & Immenroth, M. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.12.009 · Cited by 11 · see citing works
Comparing Nonrandomized Observational Studies With Randomized Controlled Trials in Cervical Disc Arthroplasty
Jee, Y. M., Bak, J. S., Weinlander, E. J., & Anderson, P. A. (2015) · Spine · doi:10.1097/brs.0000000000001377 · Cited by 11 · see citing works
Difference in surgical outcomes of rectal cancer by study design: meta-analyses of randomized clinical trials, case-matched studies, and cohort studies
Hoshino, N., Sakamoto, T., Hida, K., Takahashi, Y., Okada, H., Obama, K., & Nakayama, T. (2021) · BJS Open · doi:10.1093/bjsopen/zraa067 · Cited by 11 · see citing works
PCORI: odd name, important job, potential trouble
Kamerow, D. B. (2011) · BMJ · doi:10.1136/bmj.d2635 · Cited by 10 · see citing works
Efficacy and effectiveness of valsartan/amlodipine and valsartan/amlodipine/hydrochlorothiazide in hypertension: randomized controlled versus observational studies
Sison, J., Vega, R. M. R., Da-yi, H., Bader, G., & Brunel, P. (2017) · Current Medical Research and Opinion · doi:10.1080/03007995.2017.1412682 · Cited by 10 · see citing works
A failed attempt at developing a search filter for systematic review methodology articles in Ovid Embase
Neilson, C. J., & Lê, M. (2019) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2019.519 · Cited by 9 · see citing works
Conservative management of early-onset severe preeclampsia: comparison between randomized and observational studies a systematic review
Gracia, P. V., & Ludmir, J. (2020) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.1080/14767058.2020.1814249 · Cited by 9 · see citing works
Similarities and differences between study designs in short‐ and long‐term outcomes of laparoscopic versus open low anterior resection for rectal cancer: A systematic review and meta‐analysis of randomized, case‐matched, and cohort studies
Hoshino, N., Fukui, Y., Hida, K., & Obama, K. (2020) · Annals of Gastroenterological Surgery · doi:10.1002/ags3.12409 · Cited by 9 · see citing works
Do disparities between populations in randomized controlled trials and the real world lead to differences in outcomes?
Gray, E., Norris, S., Schmitz, S., & O’Leary, A. C. (2016) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2016-0042 · Cited by 8 · see citing works
Clinical Interventions and All-Cause Mortality of Patients with Chronic Kidney Disease: An Umbrella Systematic Review of Meta-Analyses
Kim, J. Y., Steingroever, J., Lee, K. H., Oh, J. S., Choi, M., Lee, J. M., Larkins, N., Schaefer, F. S., Hong, S. H., Jeong, G. H., et al. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9020394 · Cited by 8 · see citing works
Unknown confounders did not bias the treatment effect when improving balance of known confounders in randomized trials
Kuß, O., & Miller, M. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.012 · Cited by 7 · see citing works
Use of Non-randomised Evidence Alongside Randomised Trials in a Systematic Review of Endovascular Aneurysm Repair: Strengths and Limitations
Chambers, D., Fayter, D., Paton, F., & Woolacott, N. F. (2009) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2009.09.010 · Cited by 7 · see citing works
Comparison of All-Cause Mortality Following VTE Treatment Between Propensity Score-Adjusted Observational Studies and Matched Randomized Controlled Trials
Coscia, C., Oriol, A. J., Quezada, C. A., Muriel, A., Monréal, M., Villén, T., Barbero, E., Chiluiza, D., Yusen, R. D., & Jiménez, D. (2018) · CHEST Journal · doi:10.1016/j.chest.2018.10.016 · Cited by 7 · see citing works
Propensity score studies are unlikely to underestimate treatment effects in critical care medicine: a critical reanalysis
Kitsios, G. D. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.10.012 · Cited by 6 · see citing works
Transradial access mitigates bleeding benefit offered by bivalirudin over heparin in patients undergoing percutaneous coronary intervention: Insights from meta-analysis of randomized and observational studies
Pasala, T. K. R., Gajulapalli, R. D., Bolen, S. D., Bajaj, N. S., Gandhi, S., Tandar, A., Owan, T., & Welt, F. G. P. (2016) · International Journal of Cardiology · doi:10.1016/j.ijcard.2016.07.043 · Cited by 6 · see citing works
Advantages and Disadvantages of Trial Designs: A Review of Analysis Methods for ICD Studies
Hallstrom, A. P., Anderson, J. L., Cobb, L. A., Friedman, P. L., McAnulty, J. H., Klein, R. C., McAnulty, J. H., Steinberg, J. S., & AVID INVESTIGATORS (2000) · Pacing and Clinical Electrophysiology · doi:10.1111/j.1540-8159.2000.tb00892.x · Cited by 6 · see citing works
46th Annual SER Meeting Boston, Massachusetts June 18-21, 2013
Leung, G. M., Lam, T., Ni, M., Chan, S., Fielding, R., McDowell, I., Lee, P., Chan, B., Stewart, S., & Nan, H. (2013) · American Journal of Epidemiology · doi:10.1093/aje/kwt103 · Cited by 6 · see citing works
Commentary: does the selective inversion approach demonstrate bias in the results of studies using routinely collected data?
Sterne, J. A. C. (2018) · BMJ · doi:10.1136/bmj.k3259 · Cited by 5 · see citing works
Comparing and combining evidence of treatment effects in randomized and nonrandomized studies on the use of misoprostol to prevent postpartum hemorrhage
Morfaw, F., Miregwa, B., Ayaba, B., Mbuagbaw, L. C. E., Anderson, L. N., & Thabane, L. (2021) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12440 · Cited by 5 · see citing works
Comparison of estimated treatment effects between randomized controlled trials, case-matched, and cohort studies on laparoscopic versus open distal gastrectomy for advanced gastric cancer: a systematic review and meta-analysis
Otsuka, R., Hayashi, H., Uesato, M., Hayano, K., Murakami, K., Kano, M., Toyozumi, T., Suito, H., Matsumoto, Y., Isozaki, T., et al. (2022) · Langenbeck s Archives of Surgery · doi:10.1007/s00423-022-02454-3 · Cited by 5 · see citing works
Does observational study on the effectiveness of the Twin Blocks overestimate or underestimate the results? A comparative analysis of retrospective samples versus randomized controlled trial
Mehyar, L., Sandler, J., & Thiruvenkatachari, B. (2021) · Journal of the World Federation of Orthodontists · doi:10.1016/j.ejwf.2021.03.002 · Cited by 4 · see citing works
Bias estimation in study design: a meta-epidemiological analysis of transcatheter versus surgical aortic valve replacement
Youn, S., Wong, S. A., Chrystoja, C. C., TOMLINSON, G. A., Wijeysundera, H. C., Bell, C. M., Gagliardi, A. R., Baxter, N. N., Takata, J. L., Sandhu, L., et al. (2021) · BMC Surgery · doi:10.1186/s12893-021-01278-0 · Cited by 4 · see citing works
The effect of sodium‐glucose transport protein 2 inhibitors on mortality and heart failure in randomized trials versus observational studies
Krogh, J., Hjorthøj, C. R., Kristensen, S. L., Selmer, C., & Haugaard, S. B. (2021) · Diabetic Medicine · doi:10.1111/dme.14600 · Cited by 3 · see citing works
Tocilizumab and Mortality in Hospitalised Patients with Covid-19. A Systematic Review Comparing Randomised Trials with Observational Studies
Podmore, B., Qizilbash, N., Lacetera, A., Ubillos, I., Andresen, K., Martín, A. R., Majuelos‐Melguizo, J., Moral, A. C., Campos, M. H., Aronson, J. K., et al. (2021) · Pharmaceutics and Pharmacology Research · doi:10.31579/2693-7247/051 · Cited by 3 · see citing works
Effects of highly active antiretroviral therapy among HIV-infected patients. Results from randomised and observational studies.
Kirk, O. (2004) · PubMed · Cited by 2 · see citing works
DB3 TREATMENT EFFECT ESTIMATION:A COMPARISON BETWEEN RANDOMIZED CONTROLLED TRIALS AND OBSERVATIONAL STUDY WITH THREE DIFFERENT CAUSAL EFFECT METHODS-PROPENSITY SCORE, MARGINAL STRUCTURE MODEL AND DOUBLY ROBUST METHODS
Sun, P., Buesching, D. P., & Obenchain, R. L. (2007) · Value in Health · doi:10.1016/s1098-3015(10)68584-1 · Cited by 1 · see citing works
PUK36 - COMPARISON OF PATIENT-REPORTED OUTCOMES IN RANDOMIZED CONTROLLED TRIALS AND IN OBSERVATIONAL STUDIES FOR OVERACTIVE BLADDER: A SYSTEMATIC LITERATURE REVIEW
Borkowska, K., Taieb, V., Germain, N., Mejri, A., Jakubowska, A., & Aballéa, S. (2018) · Value in Health · doi:10.1016/j.jval.2018.09.2834 · Cited by 1 · see citing works
Inferring the effects of concurrent chemotherapy with or without adjuvant chemotherapy versus radiotherapy alone in stage II-IVB nasopharyngeal carcinoma: Comparison of results from meta-analysis of observational studies and randomized trials.
Tan, T. H., Soon, Y. Y., Cheo, T., Tey, J. C. S., & Tham, I. W. K. (2017) · Journal of Clinical Oncology · doi:10.1200/jco.2017.35.15_suppl.e17573 · Cited by 1 · see citing works
PDG1 Comparative Effectiveness and Safety of Pharmaceuticals Assessed in Observational Studies Compared with Randomized Controlled Trials
Hong, Y. D., Orsini, L. S., Guerino, J., Berger, M., Betz, G. D., Crown, W. H., Goettsch, W., Mullins, C. D., Willke, R. J., & Jansen, J. P. (2020) · Value in Health · doi:10.1016/j.jval.2020.08.684 · Cited by 1 · see citing works

Flagged first, then by citations

Research works printed but not linked: 1

Schünemann HJ, Higgins JP, Vist GE, Glasziou P, Akl EA, Skoetz N, et al. Chapter 14: Completing 'Summary of findings' tables and grading the certainty of the evidence. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook.

Sorted by citations

Dementia prevention, intervention, and care: 2024 report of the Lancet standing Commission
Livingston, G., Huntley, J., Liu, K. Y., Costafreda, S. G., Selbæk, G., Alladi, S., Ames, D., Banerjee, S., Burns, A., Brayne, C., et al. (2024) · The Lancet · doi:10.1016/s0140-6736(24)01296-0 · Cited by 3,860 · see citing works
Practical methods for incorporating summary time-to-event data into meta-analysis: updated guidance
Tierney, J. F., Burdett, S., & Fisher, D. J. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02752-z · Cited by 67 · see citing works
“Although evidence suggests that on average there is no systematic bias in meta-analyses of observational studies [24] compared with those of randomized trials, the potential remains.”
Tailoring the treatment of inflammatory rheumatic diseases by a better stratification and characterization of the clinical patient heterogeneity. Findings from a systematic literature review and experts' consensus
Ruscitti, P., Allanore, Y., Baldini, C., Barilaro, G., Bocci, E. B., Bearzi, P., Bellis, E., Berardicurti, O., Biaggi, A., Bombardieri, M., et al. (2024) · Autoimmunity Reviews · doi:10.1016/j.autrev.2024.103581 · Cited by 31 · see citing works
Effect of diabetes medications on the risk of developing dementia, mild cognitive impairment, or cognitive decline: A systematic review and meta-analysis
Hui, E. K., Mukadam, N., Kohl, G., & Livingston, G. (2025) · Journal of Alzheimer s Disease · doi:10.1177/13872877251319054 · Cited by 29 · see citing works
Treatment Effects in Randomized and Nonrandomized Studies of Pharmacological Interventions
Salcher‐Konrad, M., Nguyen, M., Savović, J., Higgins, J. P. T., & Naci, H. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.36230 · Cited by 15 · see citing works
Methodologies for network meta-analysis of randomised controlled trials in pain, anaesthesia, and perioperative medicine: a narrative review
Doleman, B., Jakobsen, J. C., Mathiesen, O., Cooper, N. J., Sutton, A. J., & Hardman, J. G. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.12.039 · Cited by 14 · see citing works
“26 Recent evidence shows that comparative observational evidence can produce similar estimates to RCTs, 27 although this practice remains controversial.”
Strength of evidence supporting cancer drug approvals in China between 2017 and 2021: a retrospective analysis
Zhang, Y., Chen, D., Fu, M., Shi, L., Naci, H., Wagner, A. K., Ross, J. S., & Guan, X. (2025) · The Lancet Oncology · doi:10.1016/s1470-2045(25)00329-8 · Cited by 10 · see citing works
Analysis of risk of bias assessments in a sample of intervention systematic reviews, Part II: focus on risk of bias tools reveals few meet current appraisal standards
Kolaski, K., Clarke, M. J., & Logan, L. R. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111460 · Cited by 9 · see citing works
Comparison of staining quality between rapid and routine hematoxylin and eosin staining of frozen breast tissue sections: an observational study
Ma, Z. Y., Zhang, X. F., Hu, Y. Z., Zhu, M., Jin, J., & Qian, P. (2024) · Journal of International Medical Research · doi:10.1177/03000605241259682 · Cited by 8 · see citing works
Evaluating agreement between individual nutrition randomised controlled trials and cohort studies - a meta-epidemiological study
Stadelmaier, J., Bantle, G., Gorenflo, L., Kiesswetter, E., Nikolakopoulou, A., & Schwingshackl, L. (2025) · BMC Medicine · doi:10.1186/s12916-025-03860-2 · Cited by 6 · see citing works
“…individual meta-analyses have evaluated how and to what extent effects from RCTs and observational studies differ when investigating a similar PI/ECO (population, intervention/exposure, comparison, outcome) question in medical research [8, 9], and recently also in the field of nutrition [10, 11].”
InfluentialSelf-citation
Congenital and Fetal Effects After Mifepristone Exposure and Continuation of Pregnancy: A Systematic Review
Turner, J., Garratt, D., Barwick, A., McLindon, L. A., Spark, M. J., & Smith, A. (2024) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3392 · Cited by 6 · see citing works
“56 Another direction may be accessing real world data from national registries such as those in the Nordic countries.”
Characterization of the literature informing health care of transgender and gender-diverse persons: A bibliometric analysis
Pattar, B. S. B., Gulamhusein, N., Rytz, C. L., Miranda, K. T., Beach, L. B., Marshall, Z., Collister, D., Greene, D. N., Whitley, C. T., Saad, N., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309169 · Cited by 6 · see citing works
Incorporating non-randomized evidence in cochrane intervention reviews: a scoping review
Axon, E., Kanellopoulou, A., Tsokani, S., Livingstone, N., Hilgart, J., & Richardson, R. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111814 · Cited by 5 · see citing works
The Effect of Non-Invasive, Non-Pharmacological Interventions on Autonomic Regulation of Cardiovascular Function in Adults with Spinal Cord Injury: A Systematic Review with Meta-Analysis
Schoffl, J., Craig, A. R., McBain, C., Pozzato, I., Middleton, J. W., & Arora, M. (2024) · Neurotrauma Reports · doi:10.1089/neur.2024.0110 · Cited by 5 · see citing works
Descriptive analysis of cochrane child-relevant systematic reviews: an update and comparison between 2009 and 2013
Crick, K., Thomson, D., Fernandes, R., Nuspl, M., Eurich, D., Rowe, B., & Hartling, L. (2017) · BMC Pediatrics · doi:10.1186/s12887-017-0908-7 · Cited by 5 · see citing works
Effectiveness of pharmacological treatments for COVID-19 due to SARS-CoV-2: a systematic literature review
García‐Vidal, C., González, J., Lumbreras, C., Salavert, M., Castro, A. A., Rubio-Rodríguez, D., & Rubio‐Terres, C. (2025) · Frontiers in Pharmacology · doi:10.3389/fphar.2025.1469681 · Cited by 4 · see citing works
Moxibustion Treatment, Alongside Conventional Western and Chinese Herbal Medical Therapies, May Improve Survival in Stage-IV Pulmonary Adenocarcinomas in a Dosage-Dependent Manner: A Prospective Observational Study With Propensity Score Analysis
Li, H., Lindberg, V., Zhu, L., Huang, X., Feng, J., & Baak, J. P. A. (2025) · Integrative Cancer Therapies · doi:10.1177/15347354251342739 · Cited by 3 · see citing works
“43 However, it is important to emphasize that independent RCTs are necessary to validate the current results.”
What Influences Women’s Knowledge, Attitudes, and Practices Toward Preconception Care? A Systematic Review and Meta-Analysis
Hidayah, N., Oktia Woro Kasmini, H., Yuniastuti, A., Ningrum, D. N. A., & Hasan, F. (2025) · F1000Research · doi:10.12688/f1000research.167200.1 · Cited by 2 · see citing works
The New Modern Medicine
Fuller, J. (2025) · doi:10.1093/9780190066178.001.0001 · Cited by 3 · see citing works
GLP‐1 Receptor Agonists in Metabolic Dysfunction‐Associated Steatotic Liver Disease: Bridging Hepatic and Cardiovascular Outcomes
Alves, G. A. M., Teranishi, M., Majumdar, R., Khan, F., Dodeja, M., Ortega, A. C. T. D. C. G., Acerboni, M., Arachchige, A. S. P. M., & James, F. (2026) · Chronic Diseases and Translational Medicine · doi:10.1002/cdt3.70048 · Cited by 2 · see citing works
“Evidence derived from routine clinical care complements randomized trials by capturing longer follow‐up, broader patient populations, and everyday prescribing patterns often excluded from strict trial eligibility criteria [53, 54].”
Back to the Roots: Safety and Tolerability of Standardised Ashwagandha (Withania somnifera) Root Extract in Healthy Adults—A Systematic Review of Biomarkers and Adverse Events
Coope, O. C., Willems, M. E. T., Levington, A., Tallon, M. J., Roman-Viñas, B., & Spurr, T. J. (2026) · Pharmaceuticals · doi:10.3390/ph19050725 · Cited by 2 · see citing works
“This heterogeneity limits the precision with which adverse event incidence can be com-https pared across trials and raises the possibility of under-reporting, particularly in open-label or non-randomised designs [60].”
Violence and Restrictive Practices Reduced by Hospital Relocation
Karstoft, L., Terkildsen, M. D., Kennedy, H. G., Jentz, C., Harpøth, A. J., Carlsen, A. H., & Sørensen, L. U. (2025) · Brain and Behavior · doi:10.1002/brb3.70760 · Cited by 2 · see citing works
Robotic pancreaticoduodenectomy in patients with overweight or obesity: a meta-analysis protocol
Yang, W., Zeng, H., & Jin, Y. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-080605 · Cited by 1 · see citing works
Single‐arm interventional versus observational studies for assessing efficacy: A meta‐epidemiological study
Chappell, M., Watkins, D., Sanderson, A., di Ruffano, L. F., Miller, P., Fewster, H., Fitzgerald, A., Edwards, M., & McCool, R. (2025) · Cochrane evidence synthesis and methods · doi:10.1002/cesm.70016 · Cited by 2 · see citing works
“Other authors have used ratios to compare effect sizes across reviews [27].”
Short-term effects of announcing spirometric lung-age on smokers’ attitudes: results from a Tunisian real-life pilot study
Khaldi, S., Derbel, K., Ghannouchi, I., Guezguez, F., SAYHI, A., Benzarti, W., Barkous, B., Bouafia, S., Dabbebi, F. Z., Charfedi, E., et al. (2024) · Expert Review of Respiratory Medicine · doi:10.1080/17476348.2024.2390992 · Cited by 1 · see citing works
Response to “Assessment of the integrity and perfusion of colorectal anastomoses: which tools do we need?”
Catarci, M., Guadagni, S., & Masedu, F. (2024) · Updates in Surgery · doi:10.1007/s13304-024-02018-8 · Cited by 1 · see citing works
Randomiserede versus observationelle studier til undersøgelse af effekten af interventioner
Sørensen, H. T. (2025) · Ugeskrift for Læger · doi:10.61409/v12240879 · Cited by 1 · see citing works
Analgesic efficacy of a laparoscopic-guided transversus abdominis plane block versus no transversus abdominis plane block in bariatric gastric bypass surgery a retrospective analysis among 332 individuals
Santo, G. D., Stumpf, O., Look, P., Abdelmalek, M., Lefering, R. L., Mantke, R., & Paasch, C. (2025) · BMC Surgery · doi:10.1186/s12893-025-02880-2 · Cited by 1 · see citing works
“These findings are largely consistent with recently published research [29].”
Valoración de la construcción a los efectos de los arts. 23.1.º LPH y 118.2 LAU 1964
Gutiérrez, E. G., & Cortés, J. C. G. (2014) · Cuaderno de arrendamientos urbanos
Essais contrôlés randomisés et études observationnelles semblent avoir la même efficacité pour fournir des données probantes en santé !
Monnin, D., & Silva, I. N. (2024) · Kinésithérapie la Revue · doi:10.1016/j.kine.2024.02.008
Koronare Herzkrankheit: Oft stummer Begleiter bei COPD
Gillissen, A. (2025) · Pneumo News · doi:10.1007/s15033-025-4127-x
Bias in Randomized Controlled Trials
Schmaling, K. B., & Kaplan, R. M. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009391733.008
Rethinking Research
Schmaling, K. B., & Kaplan, R. M. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009391733.006
Improved survival in gastric adenocarcinoma
Leijonmarck, W. (2025) · doi:10.69622/28369121
Comment on “Nebulized ketamine for acute pain management in the emergency department: A systematic review and meta-analysis”
Sanapala, K., Mehta, R., & Sah, R. (2025) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2025.07.031
Efficacy and safety of different fixation methods for acute syndesmosis injuries: protocol for a network meta-analysis of randomised and observational studies
Shen, W., Tian, D., Xue, Y., Shi, J., Deng, X., Pu, Z., & Gao, Q. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-092184
“20 To better understand fixation methods for acute syndesmosis injuries, we will conduct a comprehensive NMA that incorporates more recent studies, including RCTs and observational studies.”
Leveraging Real-world Evidence for Antihypertensive Fixed-dose Combinations: A Multidimensional Narrative Review From European Industry Perspective
Mogielnicki, M., Stus, V., & Banaszak, A. (2025) · Clinical Therapeutics · doi:10.1016/j.clinthera.2025.07.024
A Decade of Change in Sample Size and Study Setting in Leading Anesthesiology Journals: A Cross-Sectional Meta-Research Study
Tajima, R., Aoki, Y., Mizobuchi, Y., Imai, R., Suzuki, Y., Kimura, T., Mimuro, S., & Nakajima, Y. (2025) · Cureus · doi:10.7759/cureus.91767
“This restriction was necessary to ensure conceptual comparability with RCTs and to minimize heterogeneity introduced by emerging study designs that differ substantially in methodology and purpose [30].”
Preface and Acknowledgments
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.002.0004
The New Medical Model
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0002
Medical Interventions and Risk
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0009
Medical Evidence and Prediction
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0008
The Nature of Chronic Diseases and Mental Disorders
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0006
Therapeutic Skepticism
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0011
Introduction
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0014
Causation and Classification of Disease
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0005
Twenty-First-Century Medicine
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0013
Theories of Contagion and Cancer
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0003
Effect Measures and the Sufficient-Component Cause Model
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.005.0001
Problems of Extrapolation
Fuller, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780190066178.003.0010
Evidence-Based Medicine
Raj, G. M. (2025) · Introduction to Basics of Pharmacology and Toxicology · doi:10.1007/978-981-95-1920-0_4
Uncovering Individual Training-Load Profiles in Young Female Footballers: Moving Beyond the Average
Arcos, A. L., Gonzalez-Artetxe, A., Beato, M., & García-Tabar, I. (2026) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.2025-0346
Association Between Medication Adherence and Quality of Life Among Tuberculosis Patients: A Study at Permata Kuningan Hospital
Anwarudin, W., Hanum, S., Marlindasari, L., Sya’bany, A. K., & Azizah, N. (2026) · Sciences of Pharmacy · doi:10.58920/sciphar0501536
Authors’ reply to “Outcomes for metastatic triple-negative breast cancer patients treated with sacituzumab govitecan in clinical studies and real-world studies: a single-arm meta-analysis”
Liang, S., Liao, W., Jiang, J., Bao, Y., & Zheng, H. (2026) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-026-04038-2
Clinical research on homeopathic preparations: protocol template for a series of systematic reviews
Loef, M., van Haselen, R. A., & Baumgartner, S. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03168-z
“This follows methodological articles and guidelines acknowledging the relevance of NRSIs in SRs [64, 65].”
Who benefits from conditional cash transfers? New evidence on child well-being from Indonesia
Finuliyah, F., Khusaini, M., Prasetyia, F., & Fazaalloh, A. M. (2026) · International Journal of Adolescence and Youth · doi:10.1080/02673843.2026.2686342
Cross-Level Inference Errors in Scientific Interpretation—From Ecological Correlation to the Control of Inference and Communication Levels
Szromek, A. R. (2026) · Publications · doi:10.3390/publications14030041 · Cited by 1 · see citing works
“The updated review by Toews et al. (2024) further supports a cautious, methodologically nuanced comparison between observational and randomized evidence.”
Implementation of high-dose glucocorticoids and opioid consumption in the first 24 hours after elective hip and knee arthroplasty: a natural experiment study of 47,317 procedures in Eastern DenmarkPreprint
Laigaard, J., Møller, M. E. E., Olsen, M. H., Overgaard, S., Mathiesen, O., & Karlsen, A. P. H. (2026) · medRxiv · doi:10.64898/2026.08.31.26361793
El proceso de investigación en el contexto de la Medicina Basada en la Evidencia
Bautista-Vanegas, F. E., Mercado, M. M. M., & Roman, C. (2026) · doi:10.62486/978-9915-9928-3-9.ch04
Administrative Claims Databases in Dentistry: Current Applications, Methodological Trends, and Evidence Gaps: A Global Scoping Review of Dataset Characteristics
Sato, M., Ono, S., Ishimaru, M., & Ida, Y. (2026) · International Journal of Dentistry · doi:10.1155/ijod/8870016
“A Cochrane review reported little to no difference in effect estimates between RCTs and observational studies; however, the differences were signi ﬁ cant when adjustment for confounding was insuf ﬁ cient [27].”

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534 · see citing works
Self-citation
Healthcare outcomes assessed with non-experimental designs compared with those assessed in randomised trials
Bero, L. A., Anglemyer, A. T., & Horváth, H. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034 · Cited by 5 · see citing works
Self-citation
Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials
Hong, Y. D., Jansen, J. P., Guerino, J., Berger, M. L., Crown, W. H., Goettsch, W., Mullins, C. D., Willke, R. J., & Orsini, L. S. (2021) · BMC Medicine · doi:10.1186/s12916-021-02176-1 · Cited by 22 · see citing works
Practice of Epidemiology Should Meta-Analyses of Interventions Include Observational Studies in Addition to Randomized Controlled Trials? A Critical Examination of Underlying Principles
Shrier, I., Boivin, J.-F., Steele, R. J., Platt, R. W., Furlan, A., Kakuma, R., Brophy, J. M., & Rossignol, M. (2006)
Diverse criteria and methods are used to compare treatment effect estimates: a scoping review
Brignardello‐Petersen, R., Carrasco‐Labra, A., Jadad, A. R., Johnston, B. C., & TOMLINSON, G. A. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.02.001 · Cited by 1 · see citing works

Authors: Toews, I., Anglemyer, A., Nyirenda, J. L., Alsaid, D., Balduzzi, S., Grummich, K., Schwingshackl, L., & Bero, L.
Affiliations: University of Freiburg (DE)
University of Otago (NZ)
The Netherlands Cancer Institute (NL)
The University of Sydney (AU)
ORCID: Toews, Ingrid
Anglemyer, Andrew
Nyirenda, John LZ
Alsaid, Dima
Balduzzi, Sara
Grummich, Kathrin
Schwingshackl, Lukas
Bero, Lisa
Venue: Cochrane Database of Systematic Reviews
Publisher: Wiley
Published: 2024-01-04
Volume: 2024
Issue: 1
Pages: MR000034-MR000034
ISSN: 1361-6137 (ISSN-L)
1465-1858
DOI: 10.1002/14651858.mr000034.pub3
PMC: PMC10765475
OpenAlex: W4390587972
Cited by: 60
References: 168
Citations by year: 2026: 11 · 2025: 35 · 2024: 12 · 2017: 1 · 2014: 1
FWCI: 16.19
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Licence: other-oa
Locations: Cochrane Database of Systematic Reviews (publishedVersion)
PubMed Central (publishedVersion, other-oa)
PubMed (publishedVersion)
Funders: National Natural Science Foundation of China (71974137)
Renji Hospital
Health Resources and Services Administration
National Cancer Institute (P20CAS7142)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: observational studies, randomized controlled trials, comparative effectiveness research, study design bias, propensity score adjustment, heterogeneity, relative risk, odds ratio, hazard ratio, standardized mean difference, meta-epidemiology, evidence synthesis, real-world effectiveness, cohort studies, case-control study, cross-sectional study
MeSH terms: Systematic Reviews as Topic, Delivery of Health Care, Humans, Bias, Case-Control Studies, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Observational Studies as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks