Network meta-analysis: the highest level of medical evidence?
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Network meta-analyses synthesise networks of direct and indirect comparisons of interventions, and enable researchers to simultaneously assess the effects of more than two interventions for the same condition.1 Indirect evidence refers to estimates from different direct meta-analyses with a common comparator and allows for treatment comparisons that have not been directly compared in a clinical trial.1 Guidelines from the National Institute for Health and Care Excellence (NICE)2 and the Cochrane Collaboration3 typically prefer direct evidence from randomised clinical trials and conventional meta-analyses to indirect evidence. However, the WHO have recently begun using network meta-analyses to inform clinical guidelines,4 the global scientific production of network meta-analyses is increasing rapidly5 and some argue that the methodology should represent the highest level of evidence for instructing clinical decision-making.6 Are we witnessing a shift in what constitutes the highest level of medical evidence? As network meta-analyses are becoming more influential in informing clinicians and decision-makers, we need a thorough discussion of reporting standards and methodological concerns. A previous paper published in Evidence-Based Medicine considered several challenges related to heterogeneity and inconsistency (see box 1 for a description of inconsistency) in network meta-analyses.7 The present analysis considers five additional topics relevant to network meta-analyses: quality of evidence, statistical power, random errors, multiplicity issues and treatment rankings. Our aim is to debate methodological and reporting problems for these topics, and their relevance for evidence-based practice. A glossary of key terms used throughout the article is provided in box 1.Box 1 ### Glossary of terms (concerning network meta-analyses) #### Direct evidence Effect estimates from a head-to-head comparison (eg, randomised trial on A vs B). Conventional meta-analyses use direct comparisons from randomised trials. #### Indirect evidence Effect estimates from multiple direct comparisons that share a common comparator (eg, randomised trial on A …
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Hutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., Ioannidis, J. P. A., Straus, S., Thorlund, K., Jansen, J. P., et al. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2385 · Cited by 8,269 · cited 1×
“The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) group has published a statement for reporting in network meta-analyses.(19) Checklists like these are highly valuable in that they enable consistency of quality assessment across network meta-analyses.”
MethodPuhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812 · cited 4×
“14 Salanti and colleagues further supported the use of a contribution matrix to summarise the quality input of each trial to a given indirect estimate,(14) an approach not advocated by Puhan and colleagues at the time.(12) Recent advances by the GRADE working group further challenged the assumption that quality rating…”
BackgroundMethodInfluentialEvaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · cited 1×
“It is also vital to consider the transitivity assumption (see box 1) when assessing quality in network meta-analyses.(14) A recent meta-review found that 35 out of 77 (45%) network meta-analyses did not specify a plan on how to address possible effect modifiers, and the modifiers were inappropriately reported in 36 of…”
BackgroundConceptual and technical challenges in network meta-analysis
Cipriani, A., Higgins, J. P. T., Geddes, J. R., & Salanti, G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-2-201307160-00008 · Cited by 1,072 · cited 5×
“See also Cipriani et al’s study for a discussion of consistency.(1)”
BackgroundMethodInfluentialAdvances in the GRADE approach to rate the certainty in estimates from a network meta-analysis
Brignardello-Petersen, R., Bonner, A., Alexander, P. E., Siemieniuk, R. A., Furukawa, T. A., Rochwerg, B., Hazlewood, G. S., Alhazzani, W., Mustafa, R. A., Murad, M. H., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.005 · Cited by 753 · cited 1×
“Recent advances by the GRADE working group further challenged the assumption that quality ratings of indirect evidence are always necessary, and novel strategies were given on how to prioritise quality assessment in the presence of inconsistency.(15) Pressing questions on how to optimally assess quality in network met…”
BackgroundPower analysis for random-effects meta-analysis
Jackson, D., & Turner, R. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1240 · Cited by 619
Reporting of results from network meta-analyses: methodological systematic review
Bafeta, A., Trinquart, L., Seror, R., & Ravaud, P. (2014) · BMJ · doi:10.1136/bmj.g1741 · Cited by 220 · cited 1×
“Worryingly, Bafeta and colleagues found that 43 of 52 network meta-analyses (83%) did not report uncertainty intervals of their treatment ranking estimates.(20) In general, ranking estimates can be difficult to interpret, unless there are obvious differences between them.”
BackgroundNetwork meta-analyses should be the highest level of evidence in treatment guidelines
Leucht, S., Chaimani, A., Cipriani, A. S., Davis, J. M., Furukawa, T. A., & Salanti, G. (2016) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-016-0715-4 · Cited by 202 · cited 1×
“However, the WHO have recently begun using network meta-analyses to inform clinical guidelines,(4) the global scientific production of network meta-analyses is increasing rapidly(5) and some argue that the methodology should represent the highest level of evidence for instructing clinical decision-making.(6) Are we wi…”
MethodUncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials
Trinquart, L., Attiche, N., Bafeta, A., Porcher, R., & Ravaud, P. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2521 · Cited by 202 · cited 3×
“When network meta-analyses do not uphold the transitivity or consistency assumptions, treatment effect estimates may become biased.(32) This can consequently lead to less interpretable treatment rankings.”
BackgroundMethodInfluential
Cited by
Leichsenring, F., Steinert, C., Rabung, S., & Ioannidis, J. P. A. (2022) · World Psychiatry · doi:10.1002/wps.20941 · Cited by 485
Effects of anti-diabetic drugs on fracture risk: a systematic review and network meta-analysis
Zhang, Y.-S., Zheng, Y.-D., Yuan, Y., Chen, S.-C., & Xie, B.-C. (2021) · Frontiers in Endocrinology · doi:10.3389/fendo.2021.735824 · Cited by 117
Zhang, Y. S., Weng, W. Y., Xie, B. C., Meng, Y., Hao, Y. H., Liang, Y. M., & Zhou, Z. K. (2018) · Osteoporosis International · doi:10.1007/s00198-018-4649-8 · Cited by 93
Metadata
- Authors
- Faltinsen, E. G., Storebø, O. J., Jakobsen, J. C., Boesen, K., Lange, T., & Gluud, C.
- Venue
- BMJ Evidence-Based Medicine
- Publisher
- BMJ
- Published
- 2018-03-14
- Volume
- 23
- Issue
- 2
- Pages
- 56-59
- ISSN
- 1356-5524 (ISSN-L)
- 2515-446X
- 2515-4478
- OpenAlex
- W2790526974
- Cited by
- 91
- References
- 15
- FWCI
- 9.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Network meta-analysis: the highest level of medical evidence? https://app.cochat.ai/papers/10.1136/bmjebm-2017-110887
- The paper
- Faltinsen, E. G., Storebø, O. J., Jakobsen, J. C., Boesen, K., Lange, T., & Gluud, C. (2018). Network meta-analysis: the highest level of medical evidence? BMJ Evidence-Based Medicine. https://doi.org/10.1136/bmjebm-2017-110887
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.