Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: To examine whether the continuous updating of networks of prospectively planned randomised controlled trials (RCTs) ("living" network meta-analysis) provides strong evidence against the null hypothesis in comparative effectiveness of medical interventions earlier than the updating of conventional, pairwise meta-analysis. DESIGN: Empirical study of the accumulating evidence about the comparative effectiveness of clinical interventions. DATA SOURCES: Database of network meta-analyses of RCTs identified through searches of Medline, Embase, and the Cochrane Database of Systematic Reviews until 14 April 2015. ELIGIBILITY CRITERIA FOR STUDY SELECTION: Network meta-analyses published after January 2012 that compared at least five treatments and included at least 20 RCTs. Clinical experts were asked to identify in each network the treatment comparison of greatest clinical interest. Comparisons were excluded for which direct and indirect evidence disagreed, based on side, or node, splitting test (P<0.10). OUTCOMES AND ANALYSIS: Cumulative pairwise and network meta-analyses were performed for each selected comparison. Monitoring boundaries of statistical significance were constructed and the evidence against the null hypothesis was considered to be strong when the monitoring boundaries were crossed. A significance level was defined as α=5%, power of 90% (β=10%), and an anticipated treatment effect to detect equal to the final estimate from the network meta-analysis. The frequency and time to strong evidence was compared against the null hypothesis between pairwise and network meta-analyses. RESULTS: 49 comparisons of interest from 44 networks were included; most (n=39, 80%) were between active drugs, mainly from the specialties of cardiology, endocrinology, psychiatry, and rheumatology. 29 comparisons were informed by both direct and indirect evidence (59%), 13 by indirect evidence (27%), and 7 by direct evidence (14%). Both network and pairwise meta-analysis provided strong evidence against the null hypothesis for seven comparisons, but for an additional 10 comparisons only network meta-analysis provided strong evidence against the null hypothesis (P=0.002). The median time to strong evidence against the null hypothesis was 19 years with living network meta-analysis and 23 years with living pairwise meta-analysis (hazard ratio 2.78, 95% confidence interval 1.00 to 7.72, P=0.05). Studies directly comparing the treatments of interest continued to be published for eight comparisons after strong evidence had become evident in network meta-analysis. CONCLUSIONS: In comparative effectiveness research, prospectively planned living network meta-analyses produced strong evidence against the null hypothesis more often and earlier than conventional, pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Leucht, S., Cipriani, A., Spineli, L., Mavridis, D., Örey, D., Richter, F., Samara, M., Barbui, C., Engel, R. R., Geddes, J. R., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)60733-3 · Cited by 2,561
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600
Evaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402
Self-citationCumulative Meta-Analysis of Therapeutic Trials for Myocardial Infarction
Lau, J., Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992) · New England Journal of Medicine · doi:10.1056/nejm199207233270406 · Cited by 1,140
Conceptual and technical challenges in network meta-analysis
Cipriani, A., Higgins, J. P. T., Geddes, J. R., & Salanti, G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-2-201307160-00008 · Cited by 1,072
Self-citationHow Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838
Turner, R. M., Davey, J., Clarke, M. J., Thompson, S. G., & Higgins, J. P. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys041 · Cited by 837
Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis
Brignardello-Petersen, R., Bonner, A., Alexander, P. E., Siemieniuk, R. A., Furukawa, T. A., Rochwerg, B., Hazlewood, G. S., Alhazzani, W., Mustafa, R. A., Murad, M. H., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.005 · Cited by 753
Self-citationMeta-analysis: Potentials and promise
Egger, M., & Smith, G. D. (1997) · BMJ · doi:10.1136/bmj.315.7119.1371 · Cited by 646
Self-citationLiving Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap
Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001603 · Cited by 596
Evaluation of inconsistency in networks of interventions
Veroniki, A. A., Vasiliadis, H. S., Higgins, J. P., & Salanti, G. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys222 · Cited by 576
Self-citationIncreasing value and reducing waste in biomedical research: who's listening?
Moher, D., Glasziou, P., Chalmers, I., Nasser, M., Bossuyt, P. M. M., Korevaar, D. A., Graham, I. D., Ravaud, P., & Boutron, I. (2016) · The Lancet · doi:10.1016/s0140-6736(15)00307-4 · Cited by 462
When and how to update systematic reviews: consensus and checklist
Garner, P., Hopewell, S., Chandler, J., MacLehose, H., Schünemann, H. J., Akl, E. A., Beyene, J., Chang, S., Churchill, R., Dearness, K., et al. (2016) · BMJ · doi:10.1136/bmj.i3507 · Cited by 426
Pogue, J. M., & Yusuf, S. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00051-2 · Cited by 415
Network meta-analyses should be the highest level of evidence in treatment guidelines
Leucht, S., Chaimani, A., Cipriani, A. S., Davis, J. M., Furukawa, T. A., & Salanti, G. (2016) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-016-0715-4 · Cited by 202
Self-citationNetwork meta-analysis: a norm for comparative effectiveness?
Higgins, J. P. T., & Welton, N. J. (2015) · The Lancet · doi:10.1016/s0140-6736(15)61478-7 · Cited by 133
Introduction to BMJ Rapid Recommendations
Siemieniuk, R. A., Agoritsas, T., Macdonald, H., Guyatt, G. H., Brandt, L., & Vandvik, P. O. (2016) · BMJ · doi:10.1136/bmj.i5191 · Cited by 129 · cited 1×
“60 61 Our findings suggest that the gain in including both direct and indirect evidence when synthesising data from clinical research is substantial as it can considerably reduce the time to strong evidence against the null hypothesis.”
BackgroundUse of network meta-analysis in clinical guidelines
Kanters, S., Ford, N., Druyts, E., Thorlund, K., Mills, E. J., & Bansback, N. (2016) · Bulletin of the World Health Organization · doi:10.2471/blt.16.174326 · Cited by 121
Jones, A. P., Conroy, E., Williamson, P. R., Clarke, M., & Gamble, C. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-50 · Cited by 72
Vandvik, P. O., Brignardello-Petersen, R., & Guyatt, G. H. (2016) · BMC Medicine · doi:10.1186/s12916-016-0596-4 · Cited by 69
Clark, T., Berger, U., & Mansmann, U. (2013) · BMJ · doi:10.1136/bmj.f1135 · Cited by 68
Goudie, A. C., Sutton, A. J., Jones, D. R., & Donald, A. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.01.022 · Cited by 63
A multicomponent decision tool for prioritising the updating of systematic reviews
Takwoingi, Y., Hopewell, S., Tovey, D., & Sutton, A. J. (2013) · BMJ · doi:10.1136/bmj.f7191 · Cited by 57
Maintaining Reviews: Updates, Amendments and Feedback
Higgins, J. P., Green, S., & Scholten, R. J. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch3 · Cited by 54
A surveillance system to assess the need for updating systematic reviews
Ahmadzai, N., Newberry, S. J., Maglione, M. A., Tsertsvadze, A., Ansari, M. T., Hempel, S., Motala, A., Tsouros, S., Schneider Chafen, J. J., Shanman, R., et al. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-104 · Cited by 33
Cited by
Muka, T., Glisic, M., Milic, J., Verhoog, S., Bohlius, J., Bramer, W., Chowdhury, R., & Franco, O. H. (2020) · European Journal of Epidemiology · doi:10.1007/s10654-019-00576-5 · Cited by 660
BUGSnet: an R package to facilitate the conduct and reporting of Bayesian network Meta-analyses
Béliveau, A., Boyne, D. J., Slater, J., Brenner, D., & Arora, P. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0829-2 · Cited by 312
Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)
Fusar-Poli, P., Davies, C., Solmi, M., Brondino, N., De Micheli, A., Kotlicka-Antczak, M., Shin, J. I., & Radua, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00764 · Cited by 108
“Cumulative meta-analysis, defined as updating a meta-analysis whenever a new eligible RCT becomes available, can be used to address this[39].”
MethodProspective approaches to accumulating evidence
Thomas, J., Askie, L. M., Berlin, J. A., Elliott, J. H., Ghersi, D., Simmonds, M., Takwoingi, Y., Tierney, J. F., & Higgins, J. P. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch22 · Cited by 26
Salanti, G., & Higgins, J. P. T. (2022) · Systematic Reviews in Health Research · doi:10.1002/9781119099369.ch13
Self-citation
Metadata
- Authors
- Nikolakopoulou, A., Mavridis, D., Furukawa, T. A., Cipriani, A., Tricco, A. C., Straus, S. E., Siontis, G. C. M., Egger, M., & Salanti, G.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2018-02-28
- Volume
- 360
- Pages
- k585
- ISSN
- 1756-1833 (ISSN-L)
- 0959-8138
- OpenAlex
- W2789572026
- Cited by
- 107
- References
- 32
- FWCI
- 11.65
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.