The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Additional considerations are required when preparing a protocol for a systematic review with multiple interventions
Open accessAbstract
OBJECTIVES The number of systematic reviews that aim to compare multiple interventions using network meta-analysis is increasing. In this study, we highlight aspects of a standard systematic review protocol that may need modification when multiple interventions are to be compared. STUDY DESIGN AND SETTING We take the protocol format suggested by Cochrane for a standard systematic review as our reference and compare the considerations for a pairwise review with those required for a valid comparison of multiple interventions. We suggest new sections for protocols of systematic reviews including network meta-analyses with a focus on how to evaluate their assumptions. We provide example text from published protocols to exemplify the considerations. CONCLUSION Standard systematic review protocols for pairwise meta-analyses need extensions to accommodate the increased complexity of network meta-analysis. Our suggested modifications are widely applicable to both Cochrane and non-Cochrane systematic reviews involving network meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Hutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., Ioannidis, J. P. A., Straus, S., Thorlund, K., Jansen, J. P., et al. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2385 · Cited by 8,269 · cited 1×
“In the background section of CER protocols, authors should specify their rationale for undertaking a CER and justify their intention to use NMA (9).”
BackgroundSelf-citationSalanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766
Self-citationAnalysing data and undertaking meta‐analyses
Deeks, J. J., Higgins, J. P., Altman, D. G., & on behalf of the Cochrane Statistical Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch10 · Cited by 2,672
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“contrasting direct evidence with indirect from a specific loop (termed loop-specific approach (69)) or with indire ct from the entire network (termed node-splitting (70)) aim to identify pairwise comparisons or loops of evidence that might introduce important inconsistency in the network.”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
Self-citationConsistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 2×
“inconsistency models (71,72)) assess the potential for inconsistency in the entire network.”
BackgroundMethodSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Antidepressant Drug Effects and Depression Severity: A Patient-Level Meta-analysis
Fournier, J. C., DeRubeis, R. J., Hollon, S. D., Dimidjian, S., Amsterdam, J. D., Shelton, R. C., & Fawcett, J. (2010) · JAMA · doi:10.1001/jama.2009.1943 · Cited by 2,009
Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812
Self-citationAnalysing Data and Undertaking Meta‐Analyses
Deeks, J. J., Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch9 · Cited by 1,800
Self-citationRanking treatments in frequentist network meta-analysis works without resampling methods
Rücker, G., & Schwarzer, G. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0060-8 · Cited by 1,732 · cited 1×
“cumulative ranking curves, SUCRAs, mean ranks, or median ranks) as well as how the uncertainty of ranking will be reflected in the conclusions (47,48).”
BackgroundSalanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729
Self-citationCipriani, A., Furukawa, T. A., Salanti, G., Geddes, J. R., Higgins, J. P., Churchill, R., Watanabe, N., Nakagawa, A., Omori, I. M., McGuire, H., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60046-5 · Cited by 1,654
Self-citationSimultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · cited 3×
“It implies that one can validly compare two interventions via a connected indirect ou e involving one or more intermediate comparators (21,22).”
BackgroundSelf-citationEvaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · cited 3×
“There are statistical means to examine heterogeneity in a NMA, such as an I-square, and the estimate of the common heterogeneity variance (6,67,68).”
BackgroundMethodInfluentialSelf-citationThe Economic Burden of Depression in the United States: How Did It Change Between 1990 and 2000?
Greenberg, P. E., Kessler, R. C., Birnbaum, H. G., Leong, S. A., Lowe, S. W., Berglund, P. A., & Corey-Lisle, P. K. (2003) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v64n1211 · Cited by 1,280
Dias, S., Sutton, A. J., Ades, A. E., & Welton, N. J. (2013) · Medical Decision Making · doi:10.1177/0272989x12458724 · Cited by 1,207
White, I. R., Barrett, J. K., Jackson, D., & Higgins, J. P. T. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1045 · Cited by 1,114
Self-citationEvaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078
Self-citationConceptual and technical challenges in network meta-analysis
Cipriani, A., Higgins, J. P. T., Geddes, J. R., & Salanti, G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-2-201307160-00008 · Cited by 1,072 · cited 4×
“about its potential to violate the transitivity ass umption have been highlighted in general (17,61)”
BackgroundMethodInfluentialSelf-citationJansen, J. P., & Naci, H. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-159 · Cited by 656 · cited 2×
“As discussed above, th assumption of transitivity can be assessed by comparing the distribution of the potential effect modifiers across the direct comparisons in the network (10,46).”
BackgroundEvaluation of inconsistency in networks of interventions
Veroniki, A. A., Vasiliadis, H. S., Higgins, J. P., & Salanti, G. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys222 · Cited by 576 · cited 1×
“, in certain paths of the network) using the l oop-specific method (74,75) and the node-splitting”
MethodSelf-citationChaimani, A., & Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.57 · Cited by 521 · cited 3×
“We will also run network me ta-regression models to detect associations between study size and effect size (80).”
BackgroundMethodSelf-citationJansen, J. P., Trikalinos, T., Cappelleri, J. C., Daw, J., Andes, S., Eldessouki, R., & Salanti, G. (2014) · Value in Health · doi:10.1016/j.jval.2014.01.004 · Cited by 319 · cited 2×
“As discussed above, th assumption of transitivity can be assessed by comparing the distribution of the potential effect modifiers across the direct comparisons in the network (10,46).”
BackgroundSelf-citationGibbons, R. D., Hur, K., Brown, C. H., Davis, J. M., & Mann, J. J. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.2044 · Cited by 271
Furukawa, T. A., Salanti, G., Atkinson, L. Z., Leucht, S., Ruhe, H. G., Turner, E. H., Chaimani, A., Ogawa, Y., Takeshima, N., Hayasaka, Y., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010919 · Cited by 214
Self-citationUncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials
Trinquart, L., Attiche, N., Bafeta, A., Porcher, R., & Ravaud, P. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2521 · Cited by 202
Taylor, D. M., Cornelius, V., Smith, L., & Young, A. H. (2014) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12343 · Cited by 122
Hutton, B., Salanti, G., Chaimani, A., Caldwell, D. M., Schmid, C., Thorlund, K., Mills, E., Catalá-López, F., Turner, L., Altman, D. G., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092508 · Cited by 105
Self-citation
Cited by
Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis
Sbidian, E., Chaimani, A., Garcia-Doval, I., Do, G., Hua, C., Mazaud, C., Droitcourt, C., Hughes, C., Ingram, J. R., Naldi, L., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011535.pub2 · Cited by 433
Self-citationSchwingshackl, L., Chaimani, A., Hoffmann, G., Schwedhelm, C., & Boeing, H. (2018) · European Journal of Epidemiology · doi:10.1007/s10654-017-0352-x · Cited by 273
Self-citationSchwingshackl, L., Hoffmann, G., Iqbal, K., Schwedhelm, C., & Boeing, H. (2018) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqy151 · Cited by 147
“The present NMA was planned, conducted, and reported in adherence to standards of quality for reporting NMAs (15, 16).”
MethodInfluentialSalanti, G., & Higgins, J. P. T. (2022) · Systematic Reviews in Health Research · doi:10.1002/9781119099369.ch13
Self-citation
Metadata
- Authors
- Chaimani, A., Caldwell, D. M., Li, T., Higgins, J. P. T., & Salanti, G.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2017-03
- Volume
- 83
- Pages
- 65-74
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2576545646
- Cited by
- 158
- References
- 30
- FWCI
- 3.75
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.