Continuously updated network meta-analysis and statistical monitoring for timely decision-making
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Pairwise and network meta-analysis (NMA) are traditionally used retrospectively to assess existing evidence. However, the current evidence often undergoes several updates as new studies become available. In each update recommendations about the conclusiveness of the evidence and the need of future studies need to be made. In the context of prospective meta-analysis future studies are planned as part of the accumulation of the evidence. In this setting, multiple testing issues need to be taken into account when the meta-analysis results are interpreted. We extend ideas of sequential monitoring of meta-analysis to provide a methodological framework for updating NMAs. Based on the z-score for each network estimate (the ratio of effect size to its standard error) and the respective information gained after each study enters NMA we construct efficacy and futility stopping boundaries. A NMA treatment effect is considered conclusive when it crosses an appended stopping boundary. The methods are illustrated using a recently published NMA where we show that evidence about a particular comparison can become conclusive via indirect evidence even if no further trials address this comparison.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 1×
“Probabilities for each treatment being at each possible rank can be obtained and the surface under the cumulative ranking probabilities (SUCRAs) and their equivalent P-scores or mean ranks can be illustrated in graphs.(54,55) As these measures are based on the estimated summary effects at each update, Nikolakopoulou e…”
BackgroundSelf-citationOutlier and influence diagnostics for meta-analysis
Viechtbauer, W., & Cheung, M. W.-L. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.11 · Cited by 2,949
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“One could re-estimate the heterogeneity variance at each step of the analysis; this process would be associated with poor estimation of heterogeneity while the number of included studies is small.(36,37) To overcome this limitation we choose to inform the unknown heterogeneity parameter by predictive distributions con…”
MethodConsistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The consistency assumption is the statistical manifestation of transitivity and lies on the statistical agreement between different sources of evidence.(33) A statistical test for inconsistency can be monitored as soon as its evaluation is possible, that is when a closed loop (not composed only by multi-arm trials) is…”
BackgroundBucher, H. C., Guyatt, G. H., Griffith, L. E., & Walter, S. D. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00049-8 · Cited by 2,052
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Ranking treatments in frequentist network meta-analysis works without resampling methods
Rücker, G., & Schwarzer, G. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0060-8 · Cited by 1,732 · cited 1×
“Probabilities for each treatment being at each possible rank can be obtained and the surface under the cumulative ranking probabilities (SUCRAs) and their equivalent P-scores or mean ranks can be illustrated in graphs.(54,55) As these measures are based on the estimated summary effects at each update, Nikolakopoulou e…”
BackgroundSalanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729 · cited 3×
“However, even after careful planning, there is always the possibility of inconsistency in the assembled data.(20,52) Thus, we consider that in each update of NMA an estimation of inconsistency is included; here, we consider the cumulative performance of the loop specific approach.”
BackgroundSelf-citationSystematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · cited 1×
“Summary estimates of comparative effectiveness for all treatment options are thus obtained, including treatments that have not been compared directly in head-to-head comparisons.(20,21) In line with recent calls for comparative evidence at the time of market authorization,(22,23) Naci and O’Connor suggested the use of…”
White, I. R., Barrett, J. K., Jackson, D., & Higgins, J. P. T. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1045 · Cited by 1,114
Turner, R. M., Davey, J., Clarke, M. J., Thompson, S. G., & Higgins, J. P. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys041 · Cited by 837 · cited 3×
“For a ‘non-pharmacological versus any’ intervention comparison type and a semi-objective outcome a log-normal distribution for heterogeneity 2 LN 2:89, 1:91(2) has been recommended corresponding to 25th, 50th and 75th quantiles 1⁄4 0:12, 1⁄4 0:24, and 1⁄4 0:45 respectively (Appendix Figure 4).(39) We adopt a significa…”
BackgroundMethodJansen, J. P., & Naci, H. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-159 · Cited by 656 · cited 1×
“The justification of similarity in effect modifiers is important to ensure the plausibility of the transitivity assumption after each update of the network.(19,20,32) Throughout, we assume that the transitivity assumption is epidemiologically evaluated and deemed reasonable.”
BackgroundLiving Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap
Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001603 · Cited by 596 · cited 1×
“Development, refinement, and evaluation of appropriate statistical methodology as well as guidance over the optimal update of systematic reviews can aid the attempt of living systematic reviews and live cumulative NMA to bridge the gap between research evidence and health care practice.(4) Methodology described in thi…”
MethodPogue, J. M., & Yusuf, S. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00051-2 · Cited by 415 · cited 1×
“Several approaches have been suggested for measuring information in pairwise meta-analysis; we adopt an approach which is directly related to the precision of the meta-analytic estimates and consequently to the amount of evidence accumulated.(5,15,40,41) According to that approach the information contained within each…”
MethodA multicomponent decision tool for prioritising the updating of systematic reviews
Takwoingi, Y., Hopewell, S., Tovey, D., & Sutton, A. J. (2013) · BMJ · doi:10.1136/bmj.f7191 · Cited by 57 · cited 1×
“The Cochrane Collaboration has a policy about when a systematic review should be updated.(12) Updating meta-analysis (either because it is prospectively planned or because its result would be used to form a decision about conclusiveness) involves multiple tests as evidence accumulates and effect sizes are recalculated…”
Background
Cited by
Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)
Fusar-Poli, P., Davies, C., Solmi, M., Brondino, N., De Micheli, A., Kotlicka-Antczak, M., Shin, J. I., & Radua, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00764 · Cited by 108
“We employed methods which allow reducing the chances of inflating type-I errors (by using the alpha-spending monitoring boundaries (47)) and type II errors (by estimating the required “a priori anticipated information size” APIS), i.”
MethodA framework for prospective, adaptive meta-analysis (FAME) of aggregate data from randomised trials.
Tierney, J. F., Fisher, D. J., Vale, C. L., Burdett, S., Rydzewska, L. H., Rogozińska, E., Godolphin, P. J., White, I. R., & Parmar, M. K. B. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003629 · Cited by 31
“Although it has already been proposed that (absolute) information size should be optimised to ensure robust meta-analysis conclusions [27], and that it could be used to monitor accumulating evidence and account for multiple testing in cumulative pairwise [27,28] and network meta-analyses [29], these approaches seem to…”
MethodHow to update a living systematic review and keep it alive during a pandemic: a practical guide
Heron, L., Buitrago-Garcia, D., Ipekci, A. M., Baumann, R., Imeri, H., Salanti, G., Counotte, M. J., & Low, N. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02325-y · Cited by 26
“However, reviewers can employ methods that control the type I error if they want [16, 42].”
MethodSelf-citationHow to update a living systematic review and keep it alive during a pandemic: a practical guide
(2023) · doi:10.21203/rs.3.rs-2364994/v1
Metadata
- Authors
- Nikolakopoulou, A., Mavridis, D., Egger, M., & Salanti, G.
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2016-09-01
- Volume
- 27
- Issue
- 5
- Pages
- 1312-1330
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2508209283
- Cited by
- 50
- References
- 17
- FWCI
- 3.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.