Combining randomized and non-randomized evidence in network meta-analysis
Open accessAbstract
Non-randomized studies aim to reveal whether or not interventions are effective in real-life clinical practice, and there is a growing interest in including such evidence in the decision-making process. We evaluate existing methodologies and present new approaches to using non-randomized evidence in a network meta-analysis of randomized controlled trials (RCTs) when the aim is to assess relative treatment effects. We first discuss how to assess compatibility between the two types of evidence. We then present and compare an array of alternative methods that allow the inclusion of non-randomized studies in a network meta-analysis of RCTs: the naïve data synthesis, the design-adjusted synthesis, the use of non-randomized evidence as prior information and the use of three-level hierarchical models. We apply some of the methods in two previously published clinical examples comparing percutaneous interventions for the treatment of coronary in-stent restenosis and antipsychotics in patients with schizophrenia. We discuss in depth the advantages and limitations of each method, and we conclude that the inclusion of real-world evidence from non-randomized studies has the potential to corroborate findings from RCTs, increase precision and enhance the decision-making process. Copyright © 2017 John Wiley & Sons, Ltd.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prespecified design-adjusted analyses were done to combine randomised and non-randomised evidence, down-weighting studies with higher risk of bias.
“We did prespecified design-adjusted analyses to combine randomised and non-randomised evidence,31 in which studies with higher risk of bias were assigned less weight and studies of lower risk of bias were assigned more weight.”
Cites
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563 · see citing works
Leucht, S., Cipriani, A., Spineli, L., Mavridis, D., Örey, D., Richter, F., Samara, M., Barbui, C., Engel, R. R., Geddes, J. R., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)60733-3 · Cited by 2,561 · see citing works
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · see citing works
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · see citing works
Self-citationAnalysing Data and Undertaking Meta‐Analyses
Deeks, J. J., Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch9 · Cited by 1,800 · see citing works
Evaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · see citing works
Self-citationDias, S., Sutton, A. J., Ades, A. E., & Welton, N. J. (2013) · Medical Decision Making · doi:10.1177/0272989x12458724 · Cited by 1,207 · see citing works
Quantifying the impact of between-study heterogeneity in multivariate meta-analyses
Jackson, D., White, I. R., & Riley, R. D. (2012) · Statistics in Medicine · doi:10.1002/sim.5453 · Cited by 577 · see citing works
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534 · see citing works
Multiple-bias modelling for analysis of observational data
Greenland, S. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.00349.x · Cited by 497 · see citing works
GetReal in network meta-analysis: a review of the methodology
Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., Salanti, G., & on behalf of GetReal Methods Review Group (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1195 · Cited by 401 · see citing works
Self-citationSchünemann, H. J., Tugwell, P., Reeves, B. C., Akl, E. A., Santesso, N., Spencer, F. A., Shea, B., Wells, G., & Helfand, M. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1078 · Cited by 248 · see citing works
Valentine, J. C., & Thompson, S. G. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1064 · Cited by 130 · see citing works
Verde, P. E., & Ohmann, C. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1122 · Cited by 123 · see citing works
Furukawa, T. A., Miura, T., Chaimani, A., Leucht, S., Cipriani, A., Noma, H., Mitsuyasu, H., Kanba, S., & Salanti, G. (2016) · BMC Research Notes · doi:10.1186/s13104-016-2019-1 · Cited by 23 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Veroniki, A. A., Rios, P., Cogo, E., Straus, S. E., Finkelstein, Y., Kealey, R., Reynen, E., Soobiah, C., Thavorn, K., Hutton, B., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017248 · Cited by 222 · see citing works
Furukawa, T. A., Efthimiou, O., Weitz, E. S., Cipriani, A., Keller, M. B., Kocsis, J. H., Klein, D. N., Michalak, J., Salanti, G., Cuijpers, P., et al. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000489227 · Cited by 109 · see citing works
Self-citationAthanasiou, A., Veroniki, A. A., Efthimiou, O., Kalliala, I., Naci, H., Bowden, S., Paraskevaidi, M., Arbyn, M., Lyons, D., Martin-Hirsch, P., et al. (2022) · The Lancet Oncology · doi:10.1016/s1470-2045(22)00334-5 · Cited by 85 · see citing works
Self-citationTomlinson, A., Furukawa, T. A., Efthimiou, O., Salanti, G., De Crescenzo, F., Singh, I., & Cipriani, A. (2020) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300118 · Cited by 54 · see citing works
Self-citationWattanayingcharoenchai, R., Rattanasiri, S., Charakorn, C., Attia, J., & Thakkinstian, A. (2021) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16366 · Cited by 58 · see citing works
- Authors
- Efthimiou, O., Mavridis, D., Debray, T. P. A., Samara, M., Belger, M., Siontis, G. C. M., Leucht, S., Salanti, G., & 4, O. B. O. G. W. P.
- Affiliations
- University of Ioannina (GR)
- University Medical Center Utrecht (NL)
- Technical University of Munich (DE)
- Eli Lilly (United Kingdom) (GB)
- University Hospital of Bern (CH)
- University of Bern (CH)
- Corresponding author
- Efthimiou, Orestis
- ORCID
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2017-01-12
- Volume
- 36
- Issue
- 8
- Pages
- 1210-1226
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2575452307
- Cited by
- 183
- References
- 16
- Citations by year
- 2026: 13 · 2025: 23 · 2024: 32 · 2023: 20 · 2022: 25 · 2021: 22 · 2020: 17 · 2019: 16 · 2018: 9 · 2017: 6
- FWCI
- 11.75
- Type
- journal article
- Language
- en
- Open access
- green
- dspace.library.uu.nl
- Licence
- Locations
- Statistics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- BORIS (University Library Bern) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Open Access CRIS of the University of Bern
- Funders
- Innovative Medicines Initiative Joint Undertaking (115546)
- European Union's Seventh Framework Programme
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- network meta-analysis, randomized controlled trials, real-world evidence, prior information, percutaneous coronary intervention, coronary in-stent restenosis, antipsychotic drugs, schizophrenia, treatment effect comparison
- MeSH terms
- Network Meta-Analysis as Topic, Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Statistics as Topic, Models, Statistical, Randomized Controlled Trials as Topic, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
