Demystifying trial networks and network meta-analysis
Open accessAbstract
Networks of randomized clinical trials can be evaluated in the context of a network meta-analysis, a procedure that permits inferences into the comparative effectiveness of interventions that may or may not have been evaluated directly against each other. This approach is quickly gaining popularity among clinicians and guideline decision makers. However, certain methodological aspects are poorly understood. Here, we explain the geometry of a network, statistical and conceptual heterogeneity and incoherence, and challenges in the application and interpretation of data synthesis. These concepts are essential to make sense of a network meta-analysis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sparse direct evidence can make the analysis less reliable.
“as sparse direct evidence can make the analysis less reliable. 92”
From What exercise prescription is optimal to improve body composition and cardiorespiratory fitness in adults living with obesity? A network meta‐analysis (2021) · cites this paper as [92]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · see citing works
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693 · see citing works
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,149 · see citing works
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · see citing works
Jansen, J. P., Fleurence, R., Devine, B., Itzler, R. F., Barrett, A., Hawkins, N., Lee, K., Boersma, C., Annemans, L., & Cappelleri, J. C. (2011) · Value in Health · doi:10.1016/j.jval.2011.04.002 · Cited by 1,036 · see citing works
Ioannidis, J. P. A. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081086 · Cited by 716 · see citing works
Self-citationHow to Use an Article Reporting a Multiple Treatment Comparison Meta-analysis
Mills, E. J., Ioannidis, J. P. A., Thorlund, K., Schünemann, H. J., Puhan, M. A., & Guyatt, G. H. (2012) · JAMA · doi:10.1001/2012.jama.11228 · Cited by 361 · see citing works
Self-citationUse of Indirect and Mixed Treatment Comparisons for Technology Assessment
Sutton, A. J., Ades, A. E., Cooper, N. J., & Abrams, K. R. (2008) · PharmacoEconomics · doi:10.2165/00019053-200826090-00006 · Cited by 354 · see citing works
Empirical Evaluation of Very Large Treatment Effects of Medical Interventions
Pereira, T. V., Horwitz, R. I., & Ioannidis, J. P. A. (2012) · JAMA · doi:10.1001/jama.2012.13444 · Cited by 257 · see citing works
Self-citationIssues in Comparisons Between Meta-analyses and Large Trials
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.14.1089 · Cited by 246 · see citing works
Self-citationExploring the Geometry of Treatment Networks
Salanti, G., Kavvoura, F. K., & Ioannidis, J. P. A. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-7-200804010-00011 · Cited by 229 · see citing works
Self-citationSong, F., Xiong, T., Bhurke, S., Loke, Y. K., Sutton, A. J., Eastwood, A. J., Holland, R., Chen, Y., Glenny, A., Deeks, J. J., et al. (2011) · BMJ · doi:10.1136/bmj.d4909 · Cited by 199 · see citing works
Sample size and power considerations in network meta-analysis
Thorlund, K., & Mills, E. J. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-41 · Cited by 197 · see citing works
Self-citationPereira, T., Patsopoulos, N. A., Salanti, G., & Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.13 · Cited by 161 · see citing works
Self-citationCooper, N. J., Sutton, A. J., Lu, G., & Khunti, K. K. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.12.1269 · Cited by 152 · see citing works
NICE DSU Technical Support Document 1: Introduction to Evidence Synthesis for Decision Making
Dias, S., Welton, N. J., Sutton, A. J., & Ades, A. E. (2011) · Bristol Research (University of Bristol) · Cited by 110 · see citing works
Estimating the Power of Indirect Comparisons: A Simulation Study
Mills, E. J., Ghement, I., O’Regan, C., & Thorlund, K. (2011) · PLoS ONE · doi:10.1371/journal.pone.0016237 · Cited by 76 · see citing works
Self-citationNeglected tropical diseases: survey and geometry of randomised evidence
Kappagoda, S., & Ioannidis, J. P. A. (2012) · BMJ · doi:10.1136/bmj.e6512 · Cited by 43 · see citing works
Self-citationThorlund, K., Druyts, E., Aviña-Zubieta, J. A., Wu, P., & Mills, E. J. (2012) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2012-201574 · Cited by 41 · see citing works
Self-citationUse of Mixed Treatment Comparisons in Systematic Reviews
Coleman, C. I., Phung, O. J., Cappelleri, J. C., Baker, W. L., Kluger, J., White, C. M., & Sobieraj, D. M. (2012) · Europe PMC (PubMed Central) · Cited by 22 · see citing works
Issues in Comparisons Between Meta-analyses and Large Trials
Ioannidis, J. P. A., Cappelleri, J. C., & Lau, J. (1999) · Survey of Anesthesiology · doi:10.1097/00132586-199906000-00063 · Cited by 22 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Ranking treatments in frequentist network meta-analysis works without resampling methods
Rücker, G., & Schwarzer, G. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0060-8 · Cited by 1,732 · see citing works
Network meta-analysis: an introduction for clinicians
Rouse, B., Chaimani, A., & Li, T. (2017) · Internal and Emergency Medicine · doi:10.1007/s11739-016-1583-7 · Cited by 875 · see citing works
Approaches to interpreting and choosing the best treatments in network meta-analyses
Mbuagbaw, L., Rochwerg, B., Jaeschke, R., Heels-Andsell, D., Alhazzani, W., Thabane, L., & Guyatt, G. H. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0473-z · Cited by 693 · see citing works
Network meta-analysis: application and practice using Stata
Shim, S., Yoon, B.-H., Shin, I.-S., & Bae, J.-M. (2017) · Epidemiology and Health · doi:10.4178/epih.e2017047 · Cited by 656 · see citing works
Naci, H., & Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f5577 · Cited by 630 · see citing works
Self-citationGetReal in network meta-analysis: a review of the methodology
Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., Salanti, G., & on behalf of GetReal Methods Review Group (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1195 · Cited by 401 · see citing works
Van Spall, H. G. C., Rahman, T., Mytton, O., Ramasundarahettige, C., Ibrahim, Q., Kabali, C., Coppens, M., Brian Haynes, R., & Connolly, S. (2017) · European Journal of Heart Failure · doi:10.1002/ejhf.765 · Cited by 381 · see citing works
Naci, H., Salcher-Konrad, M., Dias, S., Blum, M. R., Sahoo, S. A., Nunan, D., & Ioannidis, J. P. A. (2019) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099921 · Cited by 326 · see citing works
Self-citationJansen, J. P., Trikalinos, T., Cappelleri, J. C., Daw, J., Andes, S., Eldessouki, R., & Salanti, G. (2014) · Value in Health · doi:10.1016/j.jval.2014.01.004 · Cited by 319 · see citing works
Uncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials
Trinquart, L., Attiche, N., Bafeta, A., Porcher, R., & Ravaud, P. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2521 · Cited by 202 · see citing works
O'Donoghue, G., Blake, C., Cunningham, C., Lennon, O., & Perrotta, C. (2021) · Obesity Reviews · doi:10.1111/obr.13137 · Cited by 190 · see citing works
Currier, B. S., Mcleod, J. C., Banfield, L., Beyene, J., Welton, N. J., D'Souza, A. C., Keogh, J. A. J., Lin, L., Coletta, G., Yang, A., et al. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2023-106807 · Cited by 179 · see citing works
Ballesio, A., Aquino, M. R. J. V., Feige, B., Johann, A. F., Kyle, S. D., Spiegelhalder, K., Lombardo, C., Rücker, G., Riemann, D., & Baglioni, C. (2018) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2017.01.006 · Cited by 172 · see citing works
Palpacuer, C., Duprez, R., Huneau, A., Locher, C., Boussageon, R., Laviolle, B., & Naudet, F. (2018) · Addiction · doi:10.1111/add.13974 · Cited by 172 · see citing works
Treatments for gestational diabetes: A systematic review and meta-analysis
Farrar, D., Simmonds, M., Bryant, M., Sheldon, T. A., Tuffnell, D., Golder, S., & Lawlor, D. A. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015557 · Cited by 163 · see citing works
Network meta-analysis: a norm for comparative effectiveness?
Higgins, J. P. T., & Welton, N. J. (2015) · The Lancet · doi:10.1016/s0140-6736(15)61478-7 · Cited by 133 · see citing works
Systematic reviews: guidance relevant for studies of older people
Shenkin, S. D., Harrison, J. K., Wilkinson, T., Dodds, R. M., & Ioannidis, J. P. A. (2017) · Age and Ageing · doi:10.1093/ageing/afx105 · Cited by 109 · see citing works
Self-citationZhang, Y. S., Weng, W. Y., Xie, B. C., Meng, Y., Hao, Y. H., Liang, Y. M., & Zhou, Z. K. (2018) · Osteoporosis International · doi:10.1007/s00198-018-4649-8 · Cited by 93 · see citing works
Florez, I. D., Veroniki, A.-A., Al Khalifah, R., Yepes-Nuñez, J. J., Sierra, J. M., Vernooij, R. W. M., Acosta-Reyes, J., Granados, C. M., Pérez-Gaxiola, G., Cuello-Garcia, C., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0207701 · Cited by 87 · see citing works
Kobayashi, H., Ohnishi, T., Nakagawa, R., & Yoshizawa, K. (2016) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.4405 · Cited by 79 · see citing works
“Although there are concerns about methodological issues of sample size, power, sources of bias and heterogeneity, for many comparisons, the network meta-analysis may yield more reliable and definitive results than would a pairwise meta-analysis (Mills et al., 2013).”
ResultThorlund, K., Druyts, E., Wu, P., Balijepalli, C., Keohane, D., & Mills, E. (2015) · Journal of the American Geriatrics Society · doi:10.1111/jgs.13395 · Cited by 79 · see citing works
Self-citationHTA methodology and value frameworks for evaluation and policy making for cell and gene therapies
Coyle, D., Durand-Zaleski, I., Farrington, J., Garrison, L., Graf von der Schulenburg, J.-M., Greiner, W., Longworth, L., Meunier, A., Moutié, A.-S., Palmer, S., et al. (2020) · The European Journal of Health Economics · doi:10.1007/s10198-020-01212-w · Cited by 71 · see citing works
Stuart, B., Maund, E., Wilcox, C., Sridharan, K., Sivaramakrishnan, G., Regas, C., Newell, D., Soulsby, I., Tang, K. F., Finlay, A. Y., et al. (2021) · British Journal of Dermatology · doi:10.1111/bjd.20080 · Cited by 48 · see citing works
Bennett, K., Manassis, K., Duda, S., Bagnell, A., Bernstein, G. A., Garland, E. J., Miller, L. D., Newton, A., Thabane, L., & Wilansky, P. (2015) · Depression and Anxiety · doi:10.1002/da.22400 · Cited by 45 · see citing works
Sobieraj, D. M., Coleman, C. I., Pasupuleti, V., Deshpande, A., Kaw, R., & Hernandez, A. V. (2015) · Thrombosis Research · doi:10.1016/j.thromres.2015.02.032 · Cited by 42 · see citing works
Conceptos básicos del metaanálisis en red
Catalá-López, F., Tobías, A., & Roqué, M. (2014) · Atención Primaria · doi:10.1016/j.aprim.2014.01.006 · Cited by 21 · see citing works
- Authors
- Mills, E. J., Thorlund, K., & Ioannidis, J. P. A.
- Affiliations
- University of Ottawa (CA)
- Stanford University (US)
- Stanford Medicine (US)
- Corresponding author
- Mills, E. J.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2013-05-14
- Volume
- 346
- Issue
- may14 2
- Pages
- f2914-f2914
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2024737544
- Cited by
- 742
- References
- 22
- Citations by year
- 2026: 24 · 2025: 46 · 2024: 49 · 2023: 56 · 2022: 61 · 2021: 64 · 2020: 72 · 2019: 73 · 2018: 79 · 2017: 60 · 2016: 58 · 2015: 53 · 2014: 38 · 2013: 9
- FWCI
- 58.02
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- network meta-analysis, trial networks, randomized controlled trials, comparative effectiveness, statistical heterogeneity, conceptual heterogeneity, network geometry, data synthesis, interpretation of results
- MeSH terms
- Data Interpretation, Statistical, Female, Humans, Information Services, Male, Ontario, Research Design, Sensitivity and Specificity, Meta-Analysis as Topic, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
