CoChat

Assessing Confidence in the Results of Network Meta-Analysis (Cinema)

Open access
Nikolakopoulou, Adriani, Higgins, Julian PT, Papakonstantinou, Theodore, Chaimani, Anna, Del Giovane, Cinzia, Egger, Matthias, Salanti, Georgia · 2019 · bioRxiv (Cold Spring Harbor Laboratory)
PreprintCited by 49

The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.

Abstract

Evaluation of the credibility of results from a meta-analysis has become an intrinsic part of the evidence synthesis process. We present a methodological framework to evaluate Confidence In the results from Network Meta-Analysis (CINeMA) when multiple interventions are compared. CINeMA considers six domains and we outline the methods used to form judgements about within-study bias, across-studies bias, indirectness, imprecision, heterogeneity and incoherence. Key to judgements about within-study bias and indirectness is the percentage contribution matrix, which shows how much information each study contributes to the results from network meta-analysis. The use of contribution matrix allows the semi-automation of the process, implemented in a freely available web application ( cinema.ispm.ch ). In evaluating imprecision, heterogeneity and inconsistency we consider the impact of these components of variability in forming clinical decisions. Via three examples, we show that CINeMA improves transparency and avoids the selective use of evidence when forming judgements, thus limiting subjectivity in the process. CINeMA is easy to apply even in large and complicated networks, like a network involving 18 different antidepressant drugs.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/597047

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 16 references

Research works printed but not linked: 39

  1. 10.1186/s12916-016-0764-6 · DOI · Paper page
  2. 10.1016/j.jclinepi.2010.04.026 · DOI · Paper page
  3. 2011, Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2, ValueHealth, 14, 429
  4. 2013, Evidence synthesis for decision making 7: a reviewer’s checklist, MedDecisMaking, 33, 679
  5. 2013, Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials, Med Decis Mak Int J Soc Med Decis Mak, 33, 607
  6. 10.1136/bmj.k504 · DOI · Paper page
  7. 10.1136/bmj.d5928 · DOI · Paper page
  8. 10.1016/j.jclinepi.2017.10.005 · DOI · Paper page
  9. 2018, Estimating the contribution of studies in network meta-analysis: paths, flows and streams, F1000Research, 7, 610, 10.12688/f1000research.14770.1 · DOI · Paper page
  10. 10.1371/journal.pone.0003081 · DOI · Paper page
  11. 10.1371/journal.pone.0066844 · DOI · Paper page
  12. Scherer RW , Langenberg P , von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000005. · DOI · Paper page
  13. 10.1136/bmj.f5248 · DOI · Paper page
  14. 10.1136/bmj.315.7109.640 · DOI · Paper page
  15. 10.1258/jrsm.2011.11k042 · DOI
  16. 10.1136/bmj.326.7400.1171 · DOI · Paper page
  17. 10.1056/NEJMsa065779 · DOI · Paper page
  18. 2015, Visualizing assumptions and results in network meta-analysis: The network graphs package, 15, 905
  19. Mavridis D , Efthimiou O , Leucht S , Salanti G. Publication bias and small-study effects magnified effectiveness of antipsychotics but their relative ranking remained invariant. J Clin Epidemiol. 2015 Jun 5; · DOI
  20. 10.1002/sim.5494 · DOI · Paper page
  21. 10.1016/j.jclinepi.2011.04.014 · DOI · Paper page
  22. 10.1161/CIRCOUTCOMES.111.000071 · DOI · Paper page
  23. 10.1016/j.jclinepi.2011.03.017 · DOI · Paper page
  24. 2013, Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials, Med Decis Mak Int J Soc Med Decis Mak, 33, 641
  25. 10.1002/jrsm.1045 · DOI · Paper page
  26. 10.1002/jrsm.1044 · DOI · Paper page
  27. 10.1016/j.jclinepi.2014.08.012 · DOI · Paper page
  28. 10.1198/016214505000001302 · DOI · Paper page
  29. 2009, Modeling between-trial variance structure in mixed treatment comparisons, Biostat Oxf Engl, 10, 792
  30. 10.1016/S0895-4356(97)00049-8 · DOI · Paper page
  31. Dias S , Welton NJ , Sutton AJ , Caldwell DM , Lu G , Ades AE. Inconsistency in Networks of Evidence Based on Randomised Controlled Trials [Internet]. London: National Institute for Health and Care Excellence (NICE); 2014. (NICE Decision Support Unit Technical Support Documents). Available from: http://www.ncbi.nlm.nih.gov/books/NBK310372/
  32. 10.1186/1471-2288-14-106 · DOI · Paper page
  33. 10.1186/1471-2288-12-138 · DOI · Paper page
  34. 10.1371/journal.pone.0099682 · DOI · Paper page
  35. 10.1016/j.jclinepi.2017.10.005 · DOI · Paper page
  36. 10.1136/bmj.g5630 · DOI · Paper page
  37. 2016, A threshold analysis assessed the credibility of conclusions from network meta-analysis, J Clin Epidemiol, 80, 68, 10.1016/j.jclinepi.2016.07.003 · DOI
  38. 10.1016/j.jclinepi.2016.11.002 · DOI · Paper page
  39. 10.1186/s12916-016-0764-6 · DOI · Paper page

Cited by

Metadata

Authors
Nikolakopoulou, A., Higgins, J. P., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., & Salanti, G.
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2019-04-05
OpenAlex
W2926978124
Cited by
49
References
16
Type
posted content
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.