Cited by 534Meta-analysis and systematic reviewsAdvanced Causal Inference TechniquesStatistical Methods in Epidemiology

The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials

Open access

Anglemyer, Andrew, Horvath, Hacsi T, Bero, Lisa·2014·Cochrane Database of Systematic Reviews

DOI 10.1002/14651858.mr000034.pub2PMC PMC8191367

Open this paper in CoChat — free account

Abstract

BACKGROUND: Researchers and organizations often use evidence from randomized controlled trials (RCTs) to determine the efficacy of a treatment or intervention under ideal conditions. Studies of observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and modifications of existing designs, including both randomized and observational, are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population.A systematic analysis of study design features, risk of bias, parameter interpretation, and effect size for all types of randomized and non-experimental observational studies is needed to identify specific differences in design types and potential biases. This review summarizes the results of methodological reviews that compare the outcomes of observational studies with randomized trials addressing the same question, as well as methodological reviews that compare the outcomes of different types of observational studies. OBJECTIVES: To assess the impact of study design (including RCTs versus observational study designs) on the effect measures estimated.To explore methodological variables that might explain any differences identified.To identify gaps in the existing research comparing study designs. SEARCH METHODS: We searched seven electronic databases, from January 1990 to December 2013.Along with MeSH terms and relevant keywords, we used the sensitivity-specificity balanced version of a validated strategy to identify reviews in PubMed, augmented with one term ("review" in article titles) so that it better targeted narrative reviews. No language restrictions were applied. SELECTION CRITERIA: We examined systematic reviews that were designed as methodological reviews to compare quantitative effect size estimates measuring efficacy or effectiveness of interventions tested in trials with those tested in observational studies. Comparisons included RCTs versus observational studies (including retrospective cohorts, prospective cohorts, case-control designs, and cross-sectional designs). Reviews were not eligible if they compared randomized trials with other studies that had used some form of concurrent allocation. DATA COLLECTION AND ANALYSIS: In general, outcome measures included relative risks or rate ratios (RR), odds ratios (OR), hazard ratios (HR). Using results from observational studies as the reference group, we examined the published estimates to see whether there was a relative larger or smaller effect in the ratio of odds ratios (ROR).Within each identified review, if an estimate comparing results from observational studies with RCTs was not provided, we pooled the estimates for observational studies and RCTs. Then, we estimated the ratio of ratios (risk ratio or odds ratio) for each identified review using observational studies as the reference category. Across all reviews, we synthesized these ratios to get a pooled ROR comparing results from RCTs with results from observational studies. MAIN RESULTS: Our initial search yielded 4406 unique references. Fifteen reviews met our inclusion criteria; 14 of which were included in the quantitative analysis.The included reviews analyzed data from 1583 meta-analyses that covered 228 different medical conditions. The mean number of included studies per paper was 178 (range 19 to 530).Eleven (73%) reviews had low risk of bias for explicit criteria for study selection, nine (60%) were low risk of bias for investigators' agreement for study selection, five (33%) included a complete sample of studies, seven (47%) assessed the risk of bias of their included studies,Seven (47%) reviews controlled for methodological differences between studies,Eight (53%) reviews controlled for heterogeneity among studies, nine (60%) analyzed similar outcome measures, and four (27%) were judged to be at low risk of reporting bias.Our primary quantitative analysis, including 14 reviews, showed that the pooled ROR comparing effects from RCTs with effects from observational studies was 1.08 (95% confidence interval (CI) 0.96 to 1.22). Of 14 reviews included in this analysis, 11 (79%) found no significant difference between observational studies and RCTs. One review suggested observational studies had larger effects of interest, and two reviews suggested observational studies had smaller effects of interest.Similar to the effect across all included reviews, effects from reviews comparing RCTs with cohort studies had a pooled ROR of 1.04 (95% CI 0.89 to 1.21), with substantial heterogeneity (I(2) = 68%). Three reviews compared effects of RCTs and case-control designs (pooled ROR: 1.11 (95% CI 0.91 to 1.35)).No significant difference in point estimates across heterogeneity, pharmacological intervention, or propensity score adjustment subgroups were noted. No reviews had compared RCTs with observational s

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1002/14651858.mr000034.pub2

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,232
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
Interaction revisited: the difference between two estimates.
Altman, D. G., & Bland, J. M. (2003) · BMJ · doi:10.1136/bmj.326.7382.219 · Cited by 3,203
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
A comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033
Industry sponsorship and research outcome
Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub3 · Cited by 1,697
Self-citation
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Primary total hip replacement surgery: a systematic review of outcomes and modelling of cost-effectiveness associated with different prostheses.
Fitzpatrick, R. M., Shortall, Sculpher, Murray, Morris, Lodge, Dawson, Carr, Britton, & Briggs, A. H. (1998) · Health Technology Assessment · doi:10.3310/hta2200 · Cited by 969
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919

Show all 74 references

Research works printed but not linked: 2

Deeks JJ D'Amico R Sakarovitch C Are comparability of case-mix and the use of statistical adjustment markers of quality in non-randomised studies? An empirical investigation 4th Symposium on Systematic Reviews: Pushing the Boundaries Oxford, UK 2002
Shepherd J Bagnall A Colquitt J 'Sometimes similar, sometimes different': a systematic review of meta-analyses of randomised and non-randomised policy intervention studies 14th Cochrane Colloquium, Dublin, Ireland

Sorted by citations

Items 201–300 of 532

The use of tranexamic acid reduces blood loss in osteotomy at knee level: a systematic review
Bierke, S., Häner, M., Bentzin, M., Park, H. U., & Petersen, W. J. (2022) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-022-06938-z · Cited by 21
No increased rate of cyclops lesions and extension deficits after remnant-preserving ACL reconstruction using the sparing technique
Bierke, S., Häner, M., Karpinski, K., Hees, T., & Petersen, W. J. (2022) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-022-03356-2 · Cited by 21
Silver delivery approaches in the management of partial thickness burns: A systematic review and indirect treatment comparison
Nherera, L. M., Trueman, P., Roberts, C. D., & Berg, L. T. (2017) · Wound Repair and Regeneration · doi:10.1111/wrr.12559 · Cited by 21
Telemonitoring and hemodynamic monitoring to reduce hospitalization rates in heart failure: a systematic review and meta-analysis of randomized controlled trials and real-world studies.
Tse, G., Chan, C., Gong, M., Meng, L., Zhang, J., Xl, S., Ali‐Hasan‐Al‐Saegh, S., Sawant, A. C., Bazoukis, G., Yl, X., et al. (2018) · Europe PMC (PubMed Central) · doi:10.11909/j.issn.1671-5411.2018.04.008 · Cited by 21
Post-exposure prophylaxis with hydroxychloroquine for the prevention of COVID-19, a myth or a reality? The PEP-CQ Study
Dhibar, D. P., Arora, N., Kakkar, A. K., Singla, N., Mohindra, R., Suri, V., Bhalla, A., Sharma, N., Singh, M. P., Prakash, A., et al. (2020) · International Journal of Antimicrobial Agents · doi:10.1016/j.ijantimicag.2020.106224 · Cited by 20
Biological effects of electromagnetic fields on insects: a systematic review and meta-analysis
Thill, A., Cammaerts, M., & Balmorí, A. (2023) · Reviews on Environmental Health · doi:10.1515/reveh-2023-0072 · Cited by 20
Nonrandomized studies using causal-modeling may give different answers than RCTs: a meta-epidemiological study
Ewald, H., Ioannidis, J. P. A., Ladanie, A., Cord, K. A. M., Bucher, H. C., & Hemkens, L. G. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.10.012 · Cited by 21
Cannabis and Multiple Sclerosis—The Way Forward
Rudroff, T., & Honce, J. M. (2017) · Frontiers in Neurology · doi:10.3389/fneur.2017.00299 · Cited by 20
A Bias in the Evaluation of Bias Comparing Randomized Trials with Nonexperimental Studies
Franklin, J. M., Dejene, S. Z., Huybrechts, K. F., Wang, S. V., Kulldorff, M., & Rothman, K. J. (2017) · Epidemiologic Methods · doi:10.1515/em-2016-0018 · Cited by 19
Evaluating Concordance of Bodies of Evidence from Randomized Controlled Trials, Dietary Intake, and Biomarkers of Intake in Cohort Studies: A Meta-Epidemiological Study
Beyerbach, J., Stadelmaier, J., HOFFMANN, G. E., Balduzzi, S., Bröckelmann, N., & Schwingshackl, L. (2021) · Advances in Nutrition · doi:10.1093/advances/nmab095 · Cited by 19
A standardized framework for risk-based assessment of treatment effect heterogeneity in observational healthcare databases
Rekkas, A., Klaveren, D. V., Ryan, P. B., Steyerberg, E. W., Kent, D. M., & Rijnbeek, P. R. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00794-y · Cited by 17
Clinical and radiological outcomes of a cervical cage with integrated fixation
Lonjon, N., Favreul, E., Huppert, J., Lioret, E., Delhaye, M., & Mraidi, R. (2019) · Medicine · doi:10.1097/md.0000000000014097 · Cited by 19
The Effect of Non-Gabapentinoid Anticonvulsants on Sensorimotor Recovery After Human Spinal Cord Injury
Warner, F. M., Jutzeler, C. R., Cragg, J. J., Tong, B., Grassner, L., Bradke, F., Geisler, F. H., & Kramer, J. K. (2019) · CNS Drugs · doi:10.1007/s40263-019-00622-6 · Cited by 19
The role of additional hysterectomy after concurrent chemoradiation for patients with locally advanced cervical cancer
Yoshida, K., Kajiyama, H., Yoshihara, M., Tamauchi, S., Ikeda, Y., Yoshikawa, N., Nishino, K., Niimi, K., Suzuki, S., & Kikkawa, F. (2019) · International Journal of Clinical Oncology · doi:10.1007/s10147-019-01551-6 · Cited by 19
A Controlled Clinical Study of Accelerated High-Dose Theta Burst Stimulation in Patients with Obsessive–Compulsive Disorder
Jiang, J., Wan, K. M., Liu, Y., Tang, Y., Tang, W., Liu, J., Ma, J., Xue, C., Chen, L., Qian, H., et al. (2023) · Neural Plasticity · doi:10.1155/2023/2741287 · Cited by 18
Procedural Surgical RCTs in Daily Practice
Oberkofler, C. E., Hamming, J. F., Staiger, R. D., Brosi, P., Biondo, S., Farges, O., Legemate, D. A., Morino, M., Pinna, A. D., Marques, H. P., et al. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003546 · Cited by 18
Quantifying the Child–Therapist Interaction in ASD Intervention: An Observational Coding System
Bertamini, G., Bentenuto, A., Perzolli, S., Paolizzi, E., Furlanello, C., & Venuti, P. (2021) · Brain Sciences · doi:10.3390/brainsci11030366 · Cited by 19
The PRolaCT studies — a study protocol for a combined randomised clinical trial and observational cohort study design in prolactinoma
Zandbergen, I. M., Najafabadi, A. H. Z., Pelsma, I. C. M., Marle, M. E. V. D. A., Bisschop, P. H. L. T., Boogaarts, H. D., van Bon, A. C., Burhani, B., le Cessie, S., Dekkers, O. M., et al. (2021) · Trials · doi:10.1186/s13063-021-05604-y · Cited by 18
Antidepressant Use During Acute Inpatient Care Is Associated With an Increased Risk of Psychiatric Rehospitalisation Over a 12-Month Follow-Up After Discharge
Hengartner, M. P., Passalacqua, S., Andreae, A., Heinsius, T., Hepp, U., Wulf Rössler, A., & von Wyl, A. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00079 · Cited by 17
Systematic differences in effect estimates between observational studies and randomized control trials in meta-analyses in nephrology
Kimachi, M., Ōnishi, A., Tajika, A., Kimachi, K., & Furukawa, T. A. (2021) · Scientific Reports · doi:10.1038/s41598-021-85519-5 · Cited by 18
Effects of gonadotropin-releasing hormone agonist pretreatment on frozen embryo transfer outcomes in artificial cycles: a meta-analysis
Li, X., Lin, J., Zhang, L., & Liu, Y. Z. (2022) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-022-06823-7 · Cited by 17
The Influence of Iodine-Impregnated Incision Drapes on the Bacterial Contamination of Scalpel Blades in Joint Arthroplasty
Scheidt, S., Walter, S. G., Randau, T. M., Köpf, U. S., Jordan, M. C., & Hischebeth, G. T. R. (2020) · The Journal of Arthroplasty · doi:10.1016/j.arth.2020.05.012 · Cited by 17
Survival and success of tooth–implant‐supported and solely implant‐supported double‐crown‐retained overdentures: A prospective study over a period of up to 11 years
Klotz, A., Fobbe, H., Rammelsberg, P., Bermejo, J. L., & Kappel, S. (2021) · Clinical Oral Implants Research · doi:10.1111/clr.13842 · Cited by 17
Ranking evidence in substance use and addiction
Reddon, H., Kerr, T. H., & Milloy, M. (2020) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2020.102840 · Cited by 20
Drawing causal inferences from randomized controlled trials in psychotherapy research
Kazdin, A. E. (2023) · Psychotherapy Research · doi:10.1080/10503307.2022.2130112 · Cited by 16
Efficacy gap between phase II and subsequent phase III studies in oncology
Vreman, R. A., Belitser, S. V., Mota, A. T. M., Hövels, A. M., Goettsch, W., Roes, K. C. B., Leufkens, H. G. M., & Mantel-Teeuwisse, A. K. (2020) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14237 · Cited by 16
How Generalizable Are Randomized Controlled Trials (RCTs) in Staphylococcus aureus Bacteremia? A Description of the Mortality Gap Between RCTs and Observational Studies
Bai, A. D., Lo, C. K. L., Komorowski, A. S., Suresh, M., Guo, K., Garg, A., Tandon, P., Senécal, J., Corpo, O. D., Stefanova, I. Z., et al. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac177 · Cited by 16
Can Observational Analyses of Routinely Collected Data Emulate Randomized Trials? Design and Feasibility of the Observational Patient Evidence for Regulatory Approval Science and Understanding Disease Project
Crown, W. H., Dahabreh, I. J., Li, X., Toh, S., & Bierer, B. E. (2022) · Value in Health · doi:10.1016/j.jval.2022.07.003 · Cited by 15
Perspective: The Challenge of Clinical Decision-Making for Drug Treatment in Older People. The Role of Multidimensional Assessment and Prognosis
Pilotto, A., Sancarlo, D., Daragjati, J., & Panza, F. (2015) · Frontiers in Medicine · doi:10.3389/fmed.2014.00061 · Cited by 16
Acute surgical vs non-surgical management for ocular and peri-ocular burns: a systematic review and meta-analysis
Klifto, K. M., Elhelali, A., Gurno, C. F., Seal, S. M., Asif, M., & Hultman, C. S. (2019) · Burns & Trauma · doi:10.1186/s41038-019-0161-4 · Cited by 16
From research to practice: Implementing an experimental home treatment model into routine mental health care
Stulz, N., Kawohl, W., Jäger, M., Mötteli, S., Schnyder, U., & Hepp, U. (2020) · European Psychiatry · doi:10.1192/j.eurpsy.2020.91 · Cited by 16
No inexplicable disagreements between real-world data–based nonrandomized controlled studies and randomized controlled trials were found
Mathes, T., Rombey, T., Kuß, O., & Pieper, D. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.019 · Cited by 16
A protocol for a systematic review of non-randomised evaluations of strategies to increase participant retention to randomised controlled trials
Elfeky, A., Gillies, K., Gardner, H., Fraser, C. M., & Treweek, S. P. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0696-7 · Cited by 16
A review of routinely collected data studies in urology: Methodological considerations, reporting quality, and future directions
Welk, B. K., & Kwong, J. (2017) · Canadian Urological Association Journal · doi:10.5489/cuaj.4101 · Cited by 16
What is “safety”?: Lethal means counseling as a cross-cultural communication
Pruitt, L. D., Sung, J. C., & Walker, K. A. (2022) · Military Psychology · doi:10.1080/08995605.2022.2040939 · Cited by 16
Integration of non-randomized studies with randomized controlled trials in meta-analyses of clinical studies: a meta-epidemiological study on effect estimation of interventions
Mei, F., Yao, M., Wang, Y., Huan, J., Ma, Y., Li, G., Zou, K., Li, L., & Sun, X. (2024) · BMC Medicine · doi:10.1186/s12916-024-03778-1 · Cited by 15
Comparison of Estimates between Cohort and Case–Control Studies in Meta-Analyses of Therapeutic Interventions: A Meta-Epidemiological Study
Lanza, A. L., Ravaud, P., Riveros, C., & Dechartres, A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0154877 · Cited by 15
Discharge destination and patient‐reported outcomes after inpatient treatment for isolated lower limb fractures
Kimmel, L. A., Simpson, P. M., Holland, A. E., Edwards, E. R., Cameron, P. A., de Steiger, R. N., Page, R. S., Hau, R., Bucknill, A., Kasza, J., et al. (2020) · The Medical Journal of Australia · doi:10.5694/mja2.50485 · Cited by 15
New classification of facet joint synovial cysts
Rosenstock, T., & Vajkoczy, P. (2020) · Acta Neurochirurgica · doi:10.1007/s00701-020-04264-2 · Cited by 15
Decreased mortality and increased side effects in COVID-19 patients treated with IL-6 receptor antagonists: systematic review and meta-analysis
Malgie, J., Schoones, J. W., Zeegers, M. P., & Pijls, B. G. C. W. (2021) · Scientific Reports · doi:10.1038/s41598-021-00726-4 · Cited by 15
Assessing the impact of the Royal Canadian Mounted Police (RCMP) protocol and Emotional Resilience Skills Training (ERST) among diverse public safety personnel
Carleton, R. N., McCarron, M. C. E., Krätzig, G. P., Sauer-Zavala, S. E., Neary, J. P., Lix, L. M., Fletcher, A. J., Camp, R. D., Shields, R. E., Jamshidi, L., et al. (2022) · BMC Psychology · doi:10.1186/s40359-022-00989-0 · Cited by 14
Treatment Effects in Randomized and Nonrandomized Studies of Pharmacological Interventions
Salcher‐Konrad, M., Nguyen, M., Savović, J., Higgins, J. P. T., & Naci, H. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.36230 · Cited by 15
Did the addition of concurrent chemotherapy to conventional radiotherapy improve survival for patients with HPV+ve and HPV−ve Oropharynx cancer? A population-based study
Hall, S. F., Liu, F., O’Sullivan, B., Shi, W., Rohland, S. L., Griffiths, R., & Groome, P. A. (2017) · British Journal of Cancer · doi:10.1038/bjc.2017.275 · Cited by 14
Introductory Overview of the Natural Experiments for Translation in Diabetes 2.0 (NEXT-D2) Network: Examining the Impact of US Health Policies and Practices to Prevent Diabetes and Its Complications
Duru, O. K., Mangione, C. M., Rodríguez, H. P., Ross‐Degnan, D., Wharam, J. F., Black, B. S., Kho, A., Huguet, N., Angier, H. E., Mayer, V., et al. (2018) · Current Diabetes Reports · doi:10.1007/s11892-018-0977-5 · Cited by 14
Research gaps in the philosophy of evidence‐based medicine
Mebius, A., Kennedy, A. G., & Howick, J. (2016) · Philosophy Compass · doi:10.1111/phc3.12352 · Cited by 20
Cardiovascular Safety and Superiority of Anti-Obesity Medications
Alobaida, M., Alrumayh, A., Oguntade, A. S., Al-Amodi, F., & Bwalya, M. (2021) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s311359 · Cited by 15
Routinely collected data as real-world evidence for physiotherapy practice
van Trijffel, E., Oostendorp, R. A. B., & Elvers, J. W. H. (2019) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2019.1615678 · Cited by 15
New Directions in Research on Heterogeneity of Treatment Effects for Major Depression
Luedtke, A. R., & Kessler, R. C. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4489 · Cited by 14
Estimating the effects of interventions on increasing vaccination: systematic review and meta-analysis
Liu, J., Zhang, Y., Zhang, H., & Tan, H. (2025) · BMJ Global Health · doi:10.1136/bmjgh-2024-017142 · Cited by 16
Recruitment, retention and reporting of variables related to ethnic diversity in randomised controlled trials: an umbrella review
Owusu‐Addo, E., Bennor, D. M., Orkin, A. M., Chan, A., Welch, V. A., Treweek, S. P., Green, H., Feldman, P., Ghersi, D., & Brijnath, B. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-084889 · Cited by 13
Effectiveness of treatment with biologic- and disease-modifying antirheumatic drugs in rheumatoid arthritis patients in Colombia
Machado‐Alba, J. E., Ruiz, A. F., & Machado‐Duque, M. E. (2016) · International Journal of Clinical Practice · doi:10.1111/ijcp.12809 · Cited by 13
Impact of initial myocardial perfusion imaging versus invasive coronary angiography on outcomes in coronary artery disease: a nationwide cohort study
Hung, G., Ko, K., Lin, C., Yen, R., & Kao, C. (2017) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-017-3872-4 · Cited by 13
PEER-REVIEWED JOURNAL EDITORS’ VIEWS ON REAL-WORLD EVIDENCE
Oehrlein, E. M., Graff, J. S., Perfetto, E. M., Mullins, C. D., Dubois, R. W., Anyanwu, C., & Onukwugha, E. (2018) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462317004408 · Cited by 13
Independent Housing and Support for non-homeless individuals with severe mental illness: randomised controlled trial vs. observational study – study protocol
Adamus, C., Mötteli, S., Jäger, M., & Richter, D. (2020) · BMC Psychiatry · doi:10.1186/s12888-020-02712-y · Cited by 8
The epidemiology, management and impact of surgical wounds healing by secondary intention: a research programme including the SWHSI feasibility RCT
Chetter, I. C., Arundel, C., Bell, K. J., Buckley, H., Claxton, K., Corbacho, B., Cullum, N. A., Dumville, J. C., Fairhurst, C. M., Henderson, E., et al. (2020) · Programme Grants for Applied Research · doi:10.3310/pgfar08070 · Cited by 13
Aflibercept more effectively weans patients with neovascular age-related macular degeneration off therapy compared with bevacizumab
Cao, X., Sanchez, J. C., Patel, T. P., Yang, Z., Guo, C., Malik, D., Sopeyin, A., Montaner, S., & Sodhi, A. (2022) · Journal of Clinical Investigation · doi:10.1172/jci159125 · Cited by 13
Long‐Term Hormone Replacement Therapy Is Associated with Low Coronary Artery Calcium Levels in a Cohort of Older Women: The Age, Gene/Environment Susceptibility—Reykjavik Study
Guðmundsson, A., Aspelund, T., Sigurdsson, G. T., Harris, T. B., Launer, L. J., Gudnason, V. G., & Jónsson, H. (2016) · Journal of the American Geriatrics Society · doi:10.1111/jgs.14461 · Cited by 13
Pharmacoepidemiology in older people: Purposes and future directions
Laroche, M., Sirois, C., Reeve, E., Gnjidic, D., & Morin, L. (2019) · Therapies · doi:10.1016/j.therap.2018.10.006 · Cited by 13
An in-hospital clinical care pathway with integrated decision support for cancer pain management reduced pain intensity and needs for hospital stay
Løhre, E. T., Thronæs, M., Brunelli, C., Kaasa, S., & Klepstad, P. (2019) · Supportive Care in Cancer · doi:10.1007/s00520-019-04836-8 · Cited by 12
The use and misuse of risk prediction tools for clinical decision-making
van Maaren, M. C., Hueting, T. A., Völkel, V., van Hezewijk, M., Strobbe, L. J. A., & Siesling, S. (2023) · The Breast · doi:10.1016/j.breast.2023.01.006 · Cited by 13
Neurodevelopmental Outcomes at Two Years’ Corrected Age of Very Preterm Infants after Implementation of a Post-Discharge Responsive Parenting Intervention Program (TOP Program)
Halbmeijer, N. M., Jeukens‐Visser, M., Onland, W., Flierman, M., van Kaam, A. H., & Leemhuis, A. G. (2023) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.02.025 · Cited by 12
Endovascular Thrombectomy for Large Core Volume Acute Ischemic Stroke. Updated Systematic Review and Meta-Analysis
Hukamdad, M., Biller, J., Testai, F. D., & Trifan, G. (2024) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2024.108135 · Cited by 12
Cautionary tales in the interpretation of observational studies of effects of clinical interventions
Scott, I. A., & Attia, J. (2016) · Internal Medicine Journal · doi:10.1111/imj.13167 · Cited by 12
Optimizing Use of Large Databases in Joint Arthroplasty and Orthopaedics
Browne, J. A., Springer, B. D., & Spindler, K. P. (2022) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.22.00562 · Cited by 12
Empirical evaluation of the methods used in systematic reviews including observational studies and randomized trials
Cheurfa, C., Tsokani, S., Kontouli, K., Boutron, I., & Chaimani, A. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.02.008 · Cited by 13
Non‐clinical interventions for acute respiratory infections and diarrhoeal diseases among young children in developing countries
Seguin, M., & Niño‐Zarazúa, M. (2014) · Tropical Medicine & International Health · doi:10.1111/tmi.12423 · Cited by 13
In search of justification for the unpredictability paradox
Howick, J., & Mebius, A. (2014) · Trials · doi:10.1186/1745-6215-15-480 · Cited by 7
Case studies to explore the optimal use of randomized and nonrandomized studies in evidence syntheses that use GRADE
Cuello‐García, C. A., Morgan, R. L., Brożek, J., Verbeek, J. H. A. M., Thayer, K. A., Ansari, M., Guyatt, G. H., & Schünemann, H. J. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.09.014 · Cited by 12
A Call for Computer Recess: The Impact of Computer Activities on Predominantly Minority Students’ Technology and Application Self-Efficacy
Ball, C., Huang, K., Francis, J., Kadylak, T., & Cotten, S. R. (2020) · American Behavioral Scientist · doi:10.1177/0002764220919142 · Cited by 11
Circumcision as a primary HIV preventive: Extrapolating from the available data
Howe, R. S. V. (2015) · Global Public Health · doi:10.1080/17441692.2015.1016446 · Cited by 11
Cost-effectiveness of posterior lumbar interbody fusion in the Japanese universal health insurance system
Fujimori, T., Miwa, T., Iwasaki, M., & Oda, T. (2017) · Journal of Orthopaedic Science · doi:10.1016/j.jos.2017.11.014 · Cited by 11
Highly Cited Articles in the Field of Male Infertility and Antioxidants: A Scientometric Analysis
Agarwal, A. K., Selvam, M. K. P., Baskaran, S., Finelli, R., Leisegang, K., Zenoaga-Barbăroşie, C., Pushparaj, P. N., Robert, K. A., Ambar, R. F., Iovine, C., et al. (2021) · The World Journal of Men s Health · doi:10.5534/wjmh.200181 · Cited by 11
In-house versus on-call trauma surgeon coverage: A systematic review and meta-analysis
de la Mar, A. C. J., Lokerman, R. D., Waalwijk, J. F., Ochen, Y., van der Vliet, Q. M. J., Hietbrink, F., Houwert, R. M., Leenen, L. P. H., & van Heijl, M. (2021) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000003226 · Cited by 11
Favorable safety profile of moderate hypofractionated over normofractionated radiotherapy in breast cancer patients: a multicentric prospective real-life data farming analysis
Issoufaly, I., Petit, C., Guihard, S., Eugène, R., Jung, L., Clavier, J. B., Servagi-Vernat, S., Bellefqih, S., Verret, B., Bonnet, N., et al. (2022) · Radiation Oncology · doi:10.1186/s13014-022-02044-z · Cited by 11
Comparison of statistical methods for integrating real‐world evidence in a rare events meta‐analysis of randomized controlled trials
Yao, M., Wang, Y., Ren, Y., Jia, Y., Zou, K., Li, L., & Sun, X. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1648 · Cited by 11
Real-World Evidence: Understanding Sources of Variability Through Empirical Analysis
Crown, W. H., & Bierer, B. E. (2020) · Value in Health · doi:10.1016/j.jval.2020.11.003 · Cited by 11
Independent Supported Housing for non-homeless individuals with severe mental illness: Comparison of two effectiveness studies using a randomised controlled and an observational study design
Adamus, C., Mötteli, S., Jäger, M., & Richter, D. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.1033328 · Cited by 15
Evaluating the impact of including non-randomised studies of interventions in meta-analysis of randomised controlled trials: a protocol for a meta-epidemiological study
Yao, M., Wang, Y., Busse, J. W., Briel, M., Mei, F., Li, G., Zou, K., Li, L., & Sun, X. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073232 · Cited by 11
Real world evidence for public health decision-making on vaccination policies: perspectives from an expert roundtable
de Waure, C., Gärtner, B. C., Lopalco, P. L., Puig‐Barberà, J., & Nguyen‐Van‐Tam, J. S. (2023) · Expert Review of Vaccines · doi:10.1080/14760584.2023.2290194 · Cited by 12
Big data in healthcare: are we close to it?
Naidus, E., & Celi, L. A. (2016) · Revista Brasileira de Terapia Intensiva · doi:10.5935/0103-507x.20160008 · Cited by 10
Real World Data Comparison of Standard Care vs SDM Laser Vision Protection Therapy for Prevention of Neovascular AMD
Luttrull, J. K., & Gray, G. W. (2022) · Clinical ophthalmology · doi:10.2147/opth.s366150 · Cited by 10
A systematic review and meta‐analysis of dabigatran peak and trough concentration in adults
Ng, J., Tahir, N. A. M., Chin, P. K. L., Makmor‐Bakry, M., & Saffian, S. M. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15431 · Cited by 10
Randomization, randomized trials, and analyses using observational data: A commentary on Deaton and Cartwright.
Dahabreh, I. (2018) · Social Science & Medicine (1967) · doi:10.1016/j.socscimed.2018.05.012 · Cited by 12
Survival results of 3786 patients with stage I or II laryngeal squamous cell carcinoma: a study based on a propensity score
de Carvalho, G. B., Köhler, H. F., Lira, R. B., Vartanian, J. G., & Kowalski, L. P. (2020) · Brazilian Journal of Otorhinolaryngology · doi:10.1016/j.bjorl.2020.06.007 · Cited by 10
Mortality in low-risk patients with aortic stenosis undergoing transcatheter or surgical aortic valve replacement: a reconstructed individual patient data meta-analysis
Çelik, M., Milojevic, M., Durko, A. P., Oei, F. B. S., Bogers, A. J. J. C., & Mahtab, E. A. F. (2020) · Interactive Cardiovascular and Thoracic Surgery · doi:10.1093/icvts/ivaa179 · Cited by 10
A systematic review of randomized controlled trials and reviews in the management of ventral hernias
Holihan, J. L., Nguyen, D. H., Flores-Gonzalez, J. R., Alawadi, Z. M., Nguyen, M. T., Ko, T. C., Kao, L. S., & Liang, M. K. (2016) · Journal of Surgical Research · doi:10.1016/j.jss.2016.05.009 · Cited by 10
Anaesthetic aspects in the treatment of fragility fracture patients
White, S., Foss, N. B., & Griffiths, R. (2018) · Injury · doi:10.1016/j.injury.2018.06.027 · Cited by 10
The Effects of Amplification on Listening Self-Efficacy in Adults With Sensorineural Hearing Loss
Kawaguchi, L., Wu, Y.-H., & Miller, C. W. (2019) · American Journal of Audiology · doi:10.1044/2019_aja-18-0120 · Cited by 11
Rational for meta-analysis and randomized treatment: the COVID-19 example
Raoult, D. (2020) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2020.10.012 · Cited by 10
Concordance between the results of randomized and non-randomized interventional clinical trials assessing the efficacy of drugs for COVID-19: a cross-sectional study
Shepshelovich, D., Yahav, D., Ben‐Ami, R., Goldvaser, H., & Tau, N. (2021) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkab163 · Cited by 10
The Cochrane Database of Systematic Reviews – Issue 4 2014
(2014) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12107 · Cited by 10
Clinical outcomes for grades III–V acromioclavicular dislocations favor double-button fixation compared to clavicle hook plate fixation: a systematic review and meta-analysis
Hohmann, E., & Tetsworth, K. D. (2023) · European Journal of Orthopaedic Surgery & Traumatology · doi:10.1007/s00590-023-03492-3 · Cited by 9
Outcomes after early treatment with hydroxychloroquine and azithromycin: An analysis of a database of 30,423 COVID-19 patients
Brouqui, P., Million, M., Parola, P., McCullough, P. A., & Raoult, D. (2023) · New Microbes and New Infections · doi:10.1016/j.nmni.2023.101188 · Cited by 9
Evaluating agreement between evidence from randomised controlled trials and cohort studies in nutrition: a meta-research replication study
Stadelmaier, J., Beyerbach, J., Roux, I., Harms, L., Eble, J., Nikolakopoulou, A., & Schwingshackl, L. (2024) · European Journal of Epidemiology · doi:10.1007/s10654-023-01058-5 · Cited by 9
Compliance and Toxicity of Total Neoadjuvant Therapy in Locally Advanced Rectal Cancer: A Systematic Review and Network Meta-analysis
Seow, W., Murshed, I., Bunjo, Z., Bedrikovetski, S., Stone, J., & Sammour, T. (2025) · Annals of Surgical Oncology · doi:10.1245/s10434-025-17421-7 · Cited by 10
Concordance between target trial emulation and randomised controlled trials: systematic review and meta-analysis
Wang, C. H., Tang, D., von Dadelszen, P., Ju, C., Liu, L., Wang, Y., & Magee, L. A. (2026) · BMJ · doi:10.1136/bmj-2025-086810 · Cited by 9
Efficacy and safety of rivaroxaban in real-life patients with atrial fibrillation
Barón‐Esquivias, G., Fernández‐Avilés, F., Atienza, F., Pueyo, P. P., Toro, R., & Sanmartı́n, M. (2015) · Expert Review of Cardiovascular Therapy · doi:10.1586/14779072.2015.1026259 · Cited by 9
Outcomes of 2111 COVID-19 Hospitalized Patients Treated with Hydroxychloroquine/Azithromycin and Other Regimens in Marseille, France, 2020: A Monocentric Retrospective Analysis
Lagier, J., Million, M., Cortaredona, S., Delorme, L., Colson, P., Fournier, P., Brouqui, P., Raoult, D., & Parola, P. (2022) · Therapeutics and Clinical Risk Management · doi:10.2147/tcrm.s364022 · Cited by 9
Quality of observational studies of clinical interventions: a meta-epidemiological review
Grosman, S., & Scott, I. A. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01797-1 · Cited by 9
Temporal relationship between dysthymia and temporomandibular disorder: a population-based matched case-control study in Taiwan
Lin, S. ‐., Wu, S.-L., Ko, S.-Y., Yen, C.-Y., Chiang, W.-F., & Yang, J. (2017) · BMC Oral Health · doi:10.1186/s12903-017-0343-z · Cited by 11

Authors: Anglemyer, A., Horvath, H. T., & Bero, L.
Affiliations: University of California, San Francisco (US)
University of California, San Francisco; Global Health Sciences; San Francisco California USA 94105
ORCID: Anglemyer, Andrew
Bero, Lisa
Venue: Cochrane Database of Systematic Reviews
Publisher: Wiley
Published: 2014-04-29
Volume: 2014
Issue: 4
Pages: MR000034-MR000034
ISSN: 1361-6137 (ISSN-L)
1465-1858
DOI: 10.1002/14651858.mr000034.pub2
PMC: PMC8191367
OpenAlex: W1972783815
Cited by: 534
References: 74
Citations by year: 2026: 10 · 2025: 21 · 2024: 37 · 2023: 45 · 2022: 63 · 2021: 73 · 2020: 60 · 2019: 50 · 2018: 43 · 2017: 49 · 2016: 41 · 2015: 36 · 2014: 6
FWCI: 43.06
Type: journal article
Language: en
Update notices: new_version (2024-01-04)
Open access: green
www.ncbi.nlm.nih.gov
Locations: Cochrane Database of Systematic Reviews (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: randomized controlled trials, observational studies, comparative effectiveness research, effect size estimation, risk of bias, study design comparison, relative risk, odds ratio, hazard ratio, retrospective cohort, prospective cohort, case-control study, cross-sectional study, methodological review, real-world effectiveness, bias in observational studies, effect size heterogeneity
MeSH terms: Humans, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Observational Studies as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks