CoChat
Sign Up
Cited by 220Meta-analysis and systematic reviewsDelphi Technique in ResearchClinical practice guidelines implementation

Reporting of results from network meta-analyses: methodological systematic review

Open access
Bafeta, A., Trinquart, L., Seror, R., Ravaud, P.·2014·BMJ
DOI 10.1136/bmj.g1741PMC PMC3949412
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: To examine how the results of network meta-analyses are reported. DESIGN: Methodological systematic review of published reports of network meta-analyses. DATA SOURCES: Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Medline, and Embase, searched from inception to 12 July 2012. STUDY SELECTION: All network meta-analyses comparing the clinical efficacy of three or more interventions in randomised controlled trials were included, excluding meta-analyses with an open loop network of three interventions. DATA EXTRACTION AND SYNTHESIS: The reporting of the network and results was assessed. A composite outcome included the description of the network (number of interventions, direct comparisons, and randomised controlled trials and patients for each comparison) and the reporting of effect sizes derived from direct evidence, indirect evidence, and the network meta-analysis. RESULTS: 121 network meta-analyses (55 published in general journals; 48 funded by at least one private source) were included. The network and its geometry (network graph) were not reported in 100 (83%) articles. The effect sizes derived from direct evidence, indirect evidence, and the network meta-analysis were not reported in 48 (40%), 108 (89%), and 43 (36%) articles, respectively. In 52 reports that ranked interventions, 43 did not report the uncertainty in ranking. Overall, 119 (98%) reports of network meta-analyses did not give a description of the network or effect sizes from direct evidence, indirect evidence, and the network meta-analysis. This finding did not differ by journal type or funding source. CONCLUSIONS: The results of network meta-analyses are heterogeneously reported. Development of reporting guidelines to assist authors in writing and readers in critically appraising reports of network meta-analyses is timely.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmj.g1741

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • An arm-based network meta-analysis approach allows for inclusion and analysis of a larger amount of evidence, inclusion of single-arm studies to compare against control conditions, and estimation of relative effectiveness between intervention strategies.

    “This approach allows for inclusion and analysis of a larger amount of evidence, inclusion of single-arm studies to compare against control conditions, and estimation of relative effectiveness between intervention strategies (19).”

    From Global diabetes prevention interventions: A systematic review and network meta-analysis of the real-world impact on incidence, weight, and glucose (2018) · cites this paper as [19]

    The Introduction (page unavailable in our copy) says network meta-analyses can bring together a larger evidence base and estimate relative effectiveness, including where head-to-head comparisons are lacking. It describes networks of randomised controlled trials, however, and does not support the claimed inclusion of single-arm studies or identify an arm-based approach. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. [1]Salanti G, Kavvoura FK, Ioannidis JP. Exploring the geometry of treatment networks. Ann Intern Med 2008; 148: 544-53. 18378949
  2. [2]Ioannidis JP, Karassa FB. The need to consider the wider agenda in systematic reviews and meta-analyses: breadth, timing, and depth of the evidence. BMJ 2010; 341: c4875. 20837576
  3. [3]Higgins JP, Whitehead A. Borrowing strength from external trials in a meta-analysis. Stat Med 1996; 15: 2733-49. 8981683
  4. [5]Salanti G, Higgins JP, Ades AE, Ioannidis JP. Evaluation of networks of randomized trials. Stat Methods Med Res 2008; 17: 279-301. 17925316
  5. [6]Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004; 23: 3105-24. 15449338
  6. [7]Jansen JP, Fleurence R, Devine B, Itzler R, Barrett A, Hawkins N, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health 2011; 14: 417-28. 21669366
  7. [8]Donegan S, Williamson P, Gamble C, Tudur-Smith C. Indirect comparisons: a review of reporting and methodological quality. PLoS One 2010; 5: e11054. 21085712 PMC2978085
  8. [9]Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 2009; 338: b1147. 19346285 PMC2665205
  9. [10]Bafeta A, Trinquart L, Seror R, Ravaud P. Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review. BMJ 2013; 347: f3675. 23818558 PMC3697418
  10. [12]Ades AE, Caldwell DM, Reken S, Welton NJ, Sutton AJ, Dias S. Evidence synthesis for decision making 7: a reviewer’s checklist. Med Decis Making 2013; 33: 679-91. 23804511 PMC3704204
  11. [13]Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, et al. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health 2011; 14: 429-37. 21669367
  12. [14]Edwards SJ, Clarke MJ, Wordsworth S, Borrill J. Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. Int J Clin Pract 2009; 63: 841-54. 19490195
  13. [15]Sobieraj DM, Cappelleri JC, Baker WL, Phung OJ, White CM, Coleman CI. Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review. BMJ Open 2013; 3: e003111. PMC3717466 23878173
  14. [16]Abdelhamid AS, Loke YK, Parekh-Bhurke S, Chen YF, Sutton A, Eastwood A, et al. Use of indirect comparison methods in systematic reviews: a survey of Cochrane review authors. Research Synthesis Methods 2012; 3: 71-9. 26062082
  15. [17]Song F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 2003; 326: 472. 12609941 PMC150178
  16. [18]Thorlund K, Thabane L, Mills EJ. Modelling heterogeneity variances in multiple treatment comparison meta-analysis—are informative priors the better solution? BMC Med Res Methodol 2013; 13: 2. 23311298 PMC3554418
  17. [19]Donegan S, Williamson P, D’Alessandro U, Tudur Smith C. Assessing key assumptions of network meta-analysis: a review of methods. Research Synthesis Methods 2013; 4: 291-323. 26053945
  18. [20]Ioannidis JP. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ 2009; 181: 488-93. 19654195 PMC2761440
  19. [21]Mills EJ, Thorlund K, Ioannidis JP. Demystifying trial networks and network meta-analysis. BMJ 2013; 346: f2914. 23674332
  20. [22]Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol 2011; 64: 163-71. 20688472
  21. [23]Lathyris DN, Patsopoulos NA, Salanti G, Ioannidis JP. Industry sponsorship and selection of comparators in randomized clinical trials. Eur J Clin Invest 2010; 40: 172-82. 20050879
  22. [24]Estellat C, Ravaud P. Lack of head-to-head trials and fair control arms: randomized controlled trials of biologic treatment for rheumatoid arthritis. Arch Intern Med 2012; 172: 237-44. 22332155
  23. [25]Rizos EC, Salanti G, Kontoyiannis DP, Ioannidis JP. Homophily and co-occurrence patterns shape randomized trials agendas: illustration in antifungal agents. J Clin Epidemiol 2011; 64: 830-42. 21411286
  24. [26]Stamatakis E, Weiler R, Ioannidis JP. Undue industry influences that distort healthcare research, strategy, expenditure and practice: a review. Eur J Clin Invest 2013; 43: 469-75. 23521369
  25. [27]Thorlund K, Mills EJ. Sample size and power considerations in network meta-analysis. Syst Rev 2012; 1: 41. 22992327 PMC3514119
  26. [28]Trinquart L, Abbe A, Ravaud P. Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials. PLoS One 2012; 7: e35219. 22536359 PMC3335054
  27. [29]Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials. Med Decis Making 2013; 33: 641-56. 23804508 PMC3704208
  28. [30]Song F, Xiong T, Parekh-Bhurke S, Loke YK, Sutton AJ, Eastwood AJ, et al. Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study. BMJ 2011; 343: d4909. 21846695 PMC3156578
  29. [31]Veroniki AA, Vasiliadis HS, Higgins JP, Salanti G. Evaluation of inconsistency in networks of interventions. Int J Epidemiol 2013; 42: 332-45. 23508418 PMC5411010
  30. [32]Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence-indirectness. J Clin Epidemiol 2011; 64: 1303-10. 21802903
  31. [33]Song F, Harvey I, Lilford R. Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol 2008; 61: 455-63. 18394538
  32. [34]Alkhafaji AA, Trinquart L, Baron G, Desvarieux M, Ravaud P. Impact of evergreening on patients and health insurance: a meta analysis and reimbursement cost analysis of citalopram/escitalopram antidepressants. BMC Med 2012; 10: 142. 23167972 PMC3520785
  33. [35]Madan J, Stevenson MD, Cooper KL, Ades AE, Whyte S, Akehurst R. Consistency between direct and indirect trial evidence: is direct evidence always more reliable? Value Health 2011; 14: 953-60. 21914518
  34. [36]Song F, Clark A, Bachmann MO, Maas J. Simulation evaluation of statistical properties of methods for indirect and mixed treatment comparisons. BMC Med Res Methodol 2012; 12: 138. 22970794 PMC3524036
  35. [37]Coleman CI, Phung OJ, Cappelleri JC, Baker WL, Kluger J, White CM, et al. Use of mixed treatment comparisons in systematic reviews. AHRQ Publication No. 12-EHC119-EF. Agency for Healthcare Research and Quality, 2012 (available from www.effectivehealthcare.ahrq.gov/reports/final.cfm). 23035302
  36. [38]Tan SH, Bujkiewicz S, Sutton A, Dequen P, Cooper N. Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons. J Health Serv Res Policy 2013; 18: 224-32. 23945680

Cited by

Sorted by citations

Authors
Bafeta, A., Trinquart, L., Seror, R., & Ravaud, P.
Affiliations
Inserm (FR)
Corresponding author
Bafeta, A.
ORCID
Venue
BMJ
Publisher
BMJ
Published
2014-03-11
Volume
348
Issue
mar11 5
Pages
g1741-g1741
ISSN
1756-1833 (ISSN-L)
DOI
10.1136/bmj.g1741
PMC
PMC3949412
OpenAlex
W2138280530
Cited by
220
References
2
Citations by year
2026: 16 · 2025: 21 · 2024: 25 · 2023: 20 · 2022: 13 · 2021: 9 · 2020: 25 · 2019: 19 · 2018: 32 · 2017: 19 · 2016: 12 · 2015: 6 · 2014: 3
FWCI
3.65
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • BMJ (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (submittedVersion)
Funders
  • University of Texas Rio Grande Valley
  • Fondation pour la Recherche Médicale
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
network meta-analysis, reporting quality, reporting guidelines, network graphs, effect size, direct and indirect evidence, intervention ranking, funding sources, journal types, critical appraisal
MeSH terms
Cooperative Behavior, Humans, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Evidence-Based Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review