Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 401–500 of 999
Karyotaki, E., Furukawa, T. A., Efthimiou, O., Riper, H., & Cuijpers, P. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026820 · Cited by 28
“limitations, imprecision, inconsistency, indirectness and publication bias.(51) on S etem er 2, 2019 at K yoto U nivrsity.”
BackgroundA Markov chain approach for ranking treatments in network meta‐analysis
Chaimani, A., Porcher, R., Sbidian, É., & Mavridis, D. (2020) · Statistics in Medicine · doi:10.1002/sim.8784 · Cited by 31
Self-citationDoyle, F., Freedland, K. E., Carney, R. M., de Jonge, P., Dickens, C. M., Pedersen, S. S., Sørensen, J. R., & Dempster, M. (2021) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000944 · Cited by 31
“GRADE ratings incorporate information from the study limitations, imprecision, inconsistency, indirectness, and publication bias, which can lead to downgraded quality ratings for evidence (40).”
BackgroundPascual‐Morena, C., Cavero‐Redondo, I., Álvarez‐Bueno, C., Lucerón‐Lucas‐Torres, M., Sanabria‐Martínez, G., Poyatos‐León, R., Rodríguez‐Martin, B., & Martínez‐Vizcaíno, V. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10163490 · Cited by 33
“The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) tool was used to assess the quality of evidence and make recommendations [34,35].”
MethodMethods of Weaning From Mechanical Ventilation in Adult: A Network Meta-Analysis
Jhou, H., Chen, P., Ou‐Yang, L., Lin, C. S., Tang, S.-E., & Lee, C. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.752984 · Cited by 40
Qian, Y., Fu, X., Zhang, H., Yang, Y., & Wang, G. (2023) · BMC Geriatrics · doi:10.1186/s12877-023-04239-9 · Cited by 85
“The certainty of evidence produced by the synthesis for each outcome was evaluated using the framework described by Salanti and colleagues [27] and implemented using the CINeMA (Confidence in Network Meta-Analysis) web application which allows confidence in the results to be graded as high,…”
MethodZhang, D., Ni, M., Wu, J., Liu, S., Meng, Z., Tian, J., Zhang, X., & Zhang, B. (2019) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.01470 · Cited by 30
“Network meta-analysis (NMA) can integrate direct and indirect comparisons based on clinical trials and simultaneously pool evidence on various interventions to rank their comparative efficacy and safety (Salanti et al., 2014).”
BackgroundA Practical Guide for Application of Network Meta-Analysis in Evidence Synthesis
Antoniou, S. A., Koelemay, M. J. W., Antoniou, G. A., & Mavridis, D. (2018) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2018.10.023 · Cited by 27
The development of network meta-analysis
Lee, A. (2022) · Journal of the Royal Society of Medicine · doi:10.1177/01410768221113196 · Cited by 29
Wang, S., Lan, Y., Liu, Z., Xu, S., & Wu, X. (2023) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2023.07.004 · Cited by 36
Tarp, S., Furst, D. E., Døssing, A., Østergaard, M., Lorenzen, T., Hansen, M. S., Singh, J. A., Choy, E., Boers, M., Suarez-Almazor, M. E., et al. (2016) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2016.09.003 · Cited by 30
Kishi, T., Ikuta, T., Sakuma, K., Okuya, M., & Iwata, N. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.04.032 · Cited by 27
Li, C., Liu, L., Gao, Z., Zhang, J., Chen, H., Ma, S., Liu, A., Mo, M., Wu, C., Chen, D., et al. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.693188 · Cited by 37
Fong, K. M., Au, S. Y., & Ng, G. W. Y. (2021) · Scientific Reports · doi:10.1038/s41598-021-95386-9 · Cited by 30
Marzo, V. D., Savarese, G., Tricarico, L., Hassan, S., Iacoviello, M., Porto, I., & Ameri, P. (2022) · Journal of Internal Medicine · doi:10.1111/joim.13487 · Cited by 30
Comparative diagnostic accuracy of EUS needles in solid pancreatic masses: a network meta-analysis
Han, S. S., Bhullar, F. A., Alaber, O. A., Kamal, A., Hopson, P. E., Kanthasamy, K. A., Coughlin, S., Archibugi, L., Thiruvengadam, N. R., Moreau, C., et al. (2021) · Endoscopy International Open · doi:10.1055/a-1381-7301 · Cited by 28
“The certainty of evidence in network estimates was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) ratings [25, 26].”
MethodIannone, A., Natale, P., Palmer, S. C., Nicolucci, A., Rendina, M., Giorgino, F., Laviola, L., Di Leo, A., & Strippoli, G. F. M. (2023) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.15138 · Cited by 27
“Evidence certainty was assessed using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach for network meta-analysis to critically appraise the certainty of evidence for all evaluated outcomes.(17) For this purpose, we used the Confidence in Network Meta-Analysis (CINeMA) web applic…”
MethodGe, L., Tian, J., Li, X., Song, F., Li, L., Jun, Z., Li, G., Pei, G., Qiu, X., & Yang, K. (2016) · Scientific Reports · doi:10.1038/srep37208 · Cited by 26
Filippini, G., Clerico, M., Beiki, O., Mattoscio, M., Piazza, F., Giovane, C. D., Fredrikson, S., Tramacere, I., & Scalfari, A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012200 · Cited by 27
Altheer, C., Papageorgiou, S. N., Antonarakis, G. S., & Papadopoulou, A. K. (2024) · European Journal of Orthodontics · doi:10.1093/ejo/cjae010 · Cited by 31
“…an exhaustive literature search, the systematic appraisal of the studies’ internal validity, the combination of direct and indirect data via network meta-analyses, appraisal of the quality of meta-evidence using the CINeMA approach [26] and the transparent provision of the study’s dataset [17].”
MethodHsu, T., Tsai, C., Kao, Y., Thompson, T., Carvalho, A. F., Yang, F., Tseng, P., Hsu, C.-W., Yu, C., Tu, Y., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078607 · Cited by 38
“30 We assessed the transitivity assumption by comparing the distribution of potential effect modifies across treatment comparisons.”
InfluentialSinopoulou, V., Nigam, G. B., Gordon, M. A., Ganeshan, M., Tokonyai, M. R., Dolwani, S., Iacucci, M., Rutter, M. D., Subramanian, V., Wilson, A., et al. (2024) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2024.11.008 · Cited by 34
Zhang, C., & Guan, J. (2017) · Scientometrics · doi:10.1007/s11192-017-2503-y · Cited by 28
“If older studies make a greater contribution in a network estimate, whether it is direct evidence or indirect evidence, the summary effects of that comparison will ‘‘derive most of their information from older studies’’ (Salanti et al. 2014; Chaimani and Salanti 2015).”
BackgroundWang, C., Chen, S., & Jiang, W. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.1080888 · Cited by 30
“The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach was used to evaluate the quality or certainty of each compared evidence (Salanti et al., 2014).”
MethodA threshold analysis assessed the credibility of conclusions from network meta-analysis
Caldwell, D. M., Ades, A. E., Dias, S., Watkins, S., Li, T., Taske, N., Naidoo, B., & Welton, N. J. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.07.003 · Cited by 28
“A second proposal [9] delivers an assessment of both the confidence in the pairwise contrasts and the confidence in the treatment rankings generated by the entire network of evidence.”
Self-citationInterventions for treating people with symptoms of bladder pain syndrome: a network meta-analysis
Imamura, M., SCOTT, N. W., Wallace, S. A., Ogah, J. A., Ford, A. A., Dubos, Y. A., & Brazzelli, M. G. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013325.pub2 · Cited by 32
Mavridis, D., Efthimiou, O., Leucht, S., & Salanti, G. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.05.027 · Cited by 27
Self-citationVita, G., Nöhles, V. B., Ostuzzi, G., Barbui, C., Tedeschi, F., Heuer, F. H., Keller, A., DelBello, M. P., Welge, J. A., Blom, T. J., et al. (2024) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2024.07.920 · Cited by 31
Yang, Z.-S., Lai, H., Jhou, H., Chan, W.-H., & Chen, P. (2024) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2024.111657 · Cited by 26
Ciliberto, D., Staropoli, N., Caglioti, F., Chiellino, S., Ierardi, A., Ingargiola, R., Botta, C., Arbitrio, M., Correale, P., Tassone, P., et al. (2018) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2018.03.003 · Cited by 28
Gianola, S., Castellini, G., Andreano, A., Corbetta, D., Frigerio, P., Pecoraro, V. L., Redaelli, V., Tettamanti, A., Turolla, A., Moja, L., et al. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1116-3 · Cited by 30
Alternating pressure (active) air surfaces for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013620 · Cited by 25
Comparing interventions for chronic central serous chorioretinopathy: A network meta-analysis
You, E. L., Hébert, M., Jin, T. S., Bourgault, S., Caissie, M., Tourville, É., Chen, J. L., Ordóñez-Mena, J. M., & Dirani, A. (2023) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2023.03.001 · Cited by 27
Rausa, E., Bianco, F., Kelly, M. E., Aiolfi, A., Petrelli, F., Bonitta, G., & Sgroi, G. (2019) · Journal of Surgical Oncology · doi:10.1002/jso.25410 · Cited by 25
“The confidence in estimates of the outcome was assessed using Confidence in Network Meta‐Analysis (CINeMA).(45) In accordance with Cochrane guidelines, we did not investigate publication bias as our search consider less than 10 studies for each data comparison.”
MethodBae, J. (2014) · Epidemiology and Health · doi:10.4178/epih/e2014019 · Cited by 32
An, R., Wang, D., Liang, X.-L., Chen, Q., Pang, Q., & Liu, H. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1083000 · Cited by 29
“Frontiers in Oncology 03 (heterogeneity and incoherence), indirectness and publication bias (30).”
BackgroundMahardawi, B., Jiaranuchart, S., Arunjaroensuk, S., Tompkins, K. A., Somboonsavatdee, A., & Pimkhaokham, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-39023-7 · Cited by 25
Microplastics and Soil Greenhouse Gas Emissions: A Critical Reflection on Meta-Analyses
Zhang, L., Wang, C., & Xie, B. (2025) · Environmental Science & Technology · doi:10.1021/acs.est.5c08835 · Cited by 27
Mapping the characteristics of network meta-analyses on drug therapy: A systematic review
Tonin, F. S., Steimbach, L. M., Mendes, A. M., Borba, H. H. L., Pontarolo, R., & Fernández-Llimós, F. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196644 · Cited by 26
Hao, Q., Hu, Y., Zhang, L., Ross, J., Robishaw, S., Noble, C., Wu, X., & Zhang, X. (2020) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000004795 · Cited by 24
Bouché, P.-A., Corsia, S., Nizard, R., & Resche‐Rigon, M. (2020) · The Journal of Arthroplasty · doi:10.1016/j.arth.2020.09.052 · Cited by 27
Daly, C., Mbuagbaw, L. C. E., Thabane, L., Straus, S. E., & Hamid, J. S. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01128-2 · Cited by 25
Weibel, S., Pace, N. L., Schaefer, M. S., Raj, D., Schlesinger, T., Meybohm, P., Kienbaum, P., Eberhart, L. H. J., & Kranke, P. (2021) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12429 · Cited by 27
“The 585 included RCTs comprising 97 516 randomized participants were mostly of small size with a median (IQR [range]) number of 100 (70-160 [20-5199]) participants, published between 1965 and 2017 (with 71% from 2000 onward), and primarily conducted in Asia (51%), Europe (25%), andNorthAmerica (16%).”
BackgroundChatzakis, C., Sotiriadis, A., Tsakmaki, E., Papagianni, M., Paltoglou, G., Dinas, K. D., & Mastorakos, G. P. (2021) · Nutrients · doi:10.3390/nu13072284 · Cited by 31
“Network metanalysis allows assessment of dietary supplements not directly compared in the same study [90,91] and ranks the compared interventions to identify the optimal one, based on their probability to be the most effective, as expressed by the SUCRA values [12] (References [92–103] are cited in the supplementary m…”
BackgroundSoftware to Conduct a Meta-Analysis and Network Meta-Analysis
Daly, C., & Soobiah, C. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1566-9_14 · Cited by 26
Chang, C.-L., Chen, C.-H., Yang, S. S.-D., & Chang, S.-J. (2021) · Journal of Robotic Surgery · doi:10.1007/s11701-021-01329-x · Cited by 24
“Multiarm studies which contributed more than one pairwise comparison were compared by constructing Moore–Penrose pseudoinverse matrix which allows for calculating the fitted values of the network model using a weighted least-squares approach [10].”
MethodNagendrababu, V., Faggion, C. M., Pulikkotil, S. J., Alatta, A., & Dummer, P. M. H. (2022) · International Endodontic Journal · doi:10.1111/iej.13693 · Cited by 25
Chen, L., Chen, Y., Hu, S., Lin, M., Lee, P., Chiang, A. A.-P., & Tu, Y. (2022) · Journal of Sleep Research · doi:10.1111/jsr.13686 · Cited by 26
Efficacy of surgical methods for peri-implantitis: a systematic review and network meta-analysis
Cheng, J., Chen, L., Tao, X., Qiang, X., Li, R., Ma, J., Shi, D. M., & Qiu, Z. (2023) · BMC Oral Health · doi:10.1186/s12903-023-02956-6 · Cited by 35
“ch/# doc) [37, 38], which classified the risk of bias for each study into low risk, unclear risk, or high risk of systematic errors.”
BackgroundBardideh, E., Ghorbani, M., Shafaee, H., Saeedi, P., & Younessian, F. (2023) · European Journal of Orthodontics · doi:10.1093/ejo/cjad033 · Cited by 26
Schwingshackl, L., Chaimani, A., Bechthold, A., Iqbal, K., Stelmach-Mardas, M., Hoffmann, G., Schwedhelm, C., Schlesinger, S., & Boeing, H. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0302-9 · Cited by 23
“disagreement between the different sources of evidence) in the data we will employ both local and global approaches [33].”
BackgroundSelf-citationFurukawa, T. A., Miura, T., Chaimani, A., Leucht, S., Cipriani, A., Noma, H., Mitsuyasu, H., Kanba, S., & Salanti, G. (2016) · BMC Research Notes · doi:10.1186/s13104-016-2019-1 · Cited by 23
“[3] uses weights more extensively and makes more quantitative evaluations of all the involved evidence.”
BackgroundSelf-citationYu, Q., Kong, Z., Zou, L., Chapman, R. F., Shi, Q., & Nie, J. (2023) · Journal of Exercise Science & Fitness · doi:10.1016/j.jesf.2023.09.001 · Cited by 29
Desborough, M., Hadjinicolaou, A. V., Chaimani, A., Trivella, M., Vyas, P., Doree, C. J., Hopewell, S., Stanworth, S. J., & Estcourt, L. J. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012055.pub2 · Cited by 23
Self-citationNetwork meta-analysis in health psychology and behavioural medicine: a primer
Molloy, G. J., Noone, C., Caldwell, D. M., Welton, N. J., & Newell, J. (2018) · Health Psychology Review · doi:10.1080/17437199.2018.1457449 · Cited by 25
“Useful tools for evaluating the quality of studies which have applied NMA can be found in the work of Salanti et al. (2014), Chaimani, Salanti, Leucht, Geddes, and Cipriani (2017) and Jansen et al. (2014).”
MethodSelf-citationHow does tillage intensity affect chemical soil health indicators? A United States meta‐analysis
Nunes, M. R., Karlen, D. L., Moorman, T. B., & Cambardella, C. A. (2020) · Agrosystems Geosciences & Environment · doi:10.1002/agg2.20083 · Cited by 29
Ni, M., Wang, H., Wang, M., Zhou, W., Zhang, J., Wu, J., Zhang, D., Jing, Z., Liu, X., Wu, Z., et al. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.631170 · Cited by 21
Li, X., Ni, Z.-L., Wang, J., Liu, X., Guan, H., Dai, M., Gao, X. H., Zhou, Y., Hu, X., Sun, X., et al. (2021) · Journal of Anesthesia · doi:10.1007/s00540-021-03012-9 · Cited by 29
“The certainty and hence quality of included studies in terms of within-study bias, reporting bias, indirectness, imprecision, heterogeneity, incoherence, and confidence rating was assessed according to the grading of recommendations assessment, development and evaluation (GRADE) system [19] using the CINeMA web app [2…”
MethodLi, J., Zhao, X., Zhang, Y., Wan, H., He, Y., Li, X., Yu, L., & Jin, W. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.722975 · Cited by 30
“Furthermore, the GRADE evaluation would serve as the foundation for the quality assessment, which would cover the five aspects of research restriction: study limitation, indirectness, inconsistency, imprecision, and publication bias (Salanti et al., 2014).”
MethodTsai, Y.-C., Chen, H., Lee, T.-H., Chang, H.-M., Wu, K., Chuang, C., Chang, Y.-C., Tu, Y., Hung, J., Yang, C., et al. (2021) · Cancers · doi:10.3390/cancers14010182 · Cited by 25
“Transitivity was evaluated by comparing the distribution of common comparators across different comparisons [15].”
MethodLombardo, F., Aiolfi, A., Cavalli, M., Mini, E., Lastraioli, C., Panizzo, V., Lanzaro, A., Bonitta, G., Danelli, P., Campanelli, G., et al. (2022) · Langenbeck s Archives of Surgery · doi:10.1007/s00423-021-02411-6 · Cited by 24
Chang, P., Chen, K.-H., Jhou, H., Lee, C., Chou, S., Chen, P., & Chang, T. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12040512 · Cited by 28
Fang, Y., Yue, B.-W., Ma, H., & Yuan, Y.-P. (2022) · Medicine · doi:10.1097/md.0000000000029310 · Cited by 25
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.[20]”
MethodWydall, S., Žolger, D., Owolabi, A. A., Nzekwu, B., Onwochei, D. N., & Desai, N. (2023) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-022-02389-9 · Cited by 25
Williams, T., Stein, D. J., & Ipser, J. (2018) · Evidence-Based Mental Health · doi:10.1136/eb-2017-102718 · Cited by 23
Zhang, D., Wu, J., Duan, X., Wang, K., Ni, M., Liu, S., Zhang, X., Zhang, B., & Zhao, Y. (2019) · Integrative Cancer Therapies · doi:10.1177/1534735419827098 · Cited by 24
“Compared with the double-arm meta-analysis, network meta-analysis (NMA) can synthesize multiple correlation factors, and direct or indirect comparisons simultaneously by summarizing different interventions for the same disease.(17,18) Moreover, NMA can provide evidence for identifying optimal therapies based on the ra…”
MethodResearch Methods in Learning Design and Technology
Romero‐Hall, E. (2020) · Routledge eBooks · doi:10.4324/9780429260919 · Cited by 25
Chen, L., Ferreira, M. L., Beckenkamp, P. R., Caputo, E. L., Feng, S., & Ferreira, P. H. (2020) · Physical Therapy · doi:10.1093/ptj/pzaa200 · Cited by 37
Laudani, C., Occhipinti, G., Greco, A., Spagnolo, M., Giacoppo, D., & Capodanno, D. (2025) · EuroIntervention · doi:10.4244/eij-d-24-00814 · Cited by 22
Ungprasert, P., Thongprayoon, C., & Davis, J. M. (2016) · Clinical Rheumatology · doi:10.1007/s10067-016-3204-2 · Cited by 25
“, placebo) are transitive, which means that the placebo arms are adequately similar across clinical trials [34, 35].”
BackgroundErre, G. L., Sebastiani, M., Fenu, M. A., Zinellu, A., Floris, A., Cavagna, L., Renzoni, E. A., Manfredi, A., Passiu, G., Woodman, R. J., et al. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9082560 · Cited by 25
“We assessed the level of certainty of evidence contributing to estimates for each outcome included in the NMA using the GRADE framework, rating the quality of a body of evidence on the basis of the study limitations, imprecision, inconsistency, and indirectness [16].”
MethodTorres, T. R., Barcelos, A., Filipe, P., & Fonseca, J. E. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2020.618163 · Cited by 24
“This application is based on a described methodological framework that considers six domains: within-study bias, reporting bias, indirectness, imprecision, heterogeneity, and incoherence (8).”
MethodJongkhajornpong, P., Anothaisintawee, T., Lekhanont, K., Numthavaj, P., McKay, G. J., Attia, J., & Thakkinstian, A. (2021) · Cornea · doi:10.1097/ico.0000000000002943 · Cited by 23
Sun, Y., Zhang, X., & Wang, Z. (2021) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2021.11.015 · Cited by 26
Yan, R., Cai, H., Cui, Y., Su, D., Cai, G., Lin, F., & Feng, T. (2022) · European Journal of Neurology · doi:10.1111/ene.15651 · Cited by 23
Noone, C., Dwyer, C. P., Murphy, J., Newell, J., & Molloy, G. J. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0791-9 · Cited by 29
Täger, T., Fröhlich, H., Seiz, M., Katus, H. A., & Frankenstein, L. (2019) · Heart Failure Reviews · doi:10.1007/s10741-019-09771-8 · Cited by 22
Schefft, C., Heinitz, C., Guhn, A., Brakemeier, E., Sterzer, P., & Köhler, S. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1189970 · Cited by 28
“Certainty in direct comparisons was predominantly low to moderate according to GRADE criteria; only few direct comparisons were of very low certainty while indirect comparisons were rated mostly low and very low certainty (Supplementary materials S26–S29) (39).”
Li, X., Huang, J., & Zhu, F. (2025) · Sports · doi:10.3390/sports13120415 · Cited by 28
“The certainty of evidence for each outcome was appraised with the GRADE framework (Grading of Recommendations, Assessment, Development, and Evaluation) [30].”
Naing, C. M., Whittaker, M. A., Htet, N. H., Aye, S. N., & Mak, J. W. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225882 · Cited by 24
“We assessed the quality of evidence derived from the pairwise and NMA, following the GRADE approach, as described elsewhere [15,17,21,22].”
The statistical importance of a study for a network meta-analysis estimate
Rücker, G., Nikolakopoulou, A., Papakonstantinou, T., Salanti, G., Riley, R. D., & Schwarzer, G. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01075-y · Cited by 24
Self-citationXavier, A. A. D. S., da Rosa, P. P., Mackmill, L. D. B., & Roll, V. F. B. (2020) · Research in Veterinary Science · doi:10.1016/j.rvsc.2020.11.013 · Cited by 23
Sharma, G., Pareek, T., Kaundal, P., Tyagi, S., Singh, S. P., Thummala, Y., Devana, S. K., & Sharma, A. J. (2022) · International braz j urol · doi:10.1590/s1677-5538.ibju.2020.0548 · Cited by 27
López, P., Newton, R. U., Taaffe, D. R., Singh, F., Lyons‐Wall, P., Buffart, L. M., Tang, C., Hayne, D., & Galvao, D. A. (2021) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000002843 · Cited by 23
“, based on network funnel plot and the Egger’s test) (24).”
MethodHuang, Y., Teng, T., Shen, X., Chen, S.-D., Wang, R., Zhang, R., Dou, K.-X., Zhong, X.-L., Wang, J., Chen, K., et al. (2022) · Ageing Research Reviews · doi:10.1016/j.arr.2022.101568 · Cited by 25
Li, C., Lu, F., Chen, J., Ma, J., & Xu, N. (2022) · Frontiers in Nutrition · doi:10.3389/fnut.2022.919156 · Cited by 28
“Comparisonadjusted funnel plots were used in assessing publication bias (85, 86).”
MethodJung, A., Carvalho, G. F., Szikszay, T. M., Pawlowsky, V., Gabler, T., & Luedtke, K. (2023) · Physical Therapy · doi:10.1093/ptj/pzad154 · Cited by 21
Schwingshackl, L., Chaimani, A., HOFFMANN, G. E., Schwedhelm, C., & Boeing, H. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014736 · Cited by 27
Self-citationGraphs of study contributions and covariate distributions for network meta‐regression
Donegan, S., Dias, S., Smith, C. T., Marinho, V. C., & Welton, N. J. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1292 · Cited by 21
Applying GRADE to a network meta-analysis of antidepressants led to more conservative conclusions
Bonner, A., Alexander, P. E., Brignardello‐Petersen, R., Furukawa, T. A., Siemieniuk, R. A. C., Zhang, Y., Wiercioch, W., Flórez, I. D., Fei, Y., Agarwal, A., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.05.009 · Cited by 26
Observational research on sodium glucose co‐transporter‐2 inhibitors: A real breakthrough?
Raschi, E., Poluzzi, E., Fadini, G. P., Marchesini, G., & Ponti, F. D. (2018) · Diabetes Obesity and Metabolism · doi:10.1111/dom.13468 · Cited by 24
“This systematic review with an NMA has several strengths and followed current standards for reporting and conducting meta-analytical studies.(75,76) However, clinicians should be advised that focusing on the probability of being ranked first is potentially misleading.”
MethodDoyle, F., Freedland, K. E., Carney, R. M., de Jonge, P., Dickens, C. M., Pedersen, S. S., Sørensen, J. R., & Dempster, M. (2019) · Systematic Reviews · doi:10.1186/s13643-019-0985-9 · Cited by 20
“which could downgrade the obtained summary evidence [49].”
BackgroundKim, D. K., Lee, J. Y., Jung, J. H., Hah, Y. S., Koo, K. C., Lee, K. S., Chung, B. H., & Cho, K. S. (2019) · Scientific Reports · doi:10.1038/s41598-019-41412-w · Cited by 21
Liang, J., Lü, L., Wang, Y., Jia, R., Qu, X.-Y., Li, J., Li, J., Qian, S., Qian, Y., Wang, S., et al. (2019) · Ageing Research Reviews · doi:10.1016/j.arr.2019.100965 · Cited by 25
Sargeant, J. M., Deb, B., Bergevin, M. D., Churchill, K. J., Dawkins, K., Dunn, J. N., Hu, D., Moody, C. M., O'Connor, A. M. C., O’Sullivan, T. L., et al. (2019) · Animal Health Research Reviews · doi:10.1017/s1466252319000173 · Cited by 23
“In a GRADE assessment for NMA, indirectness refers to the differences between the populations, interventions, and outcomes in the included studies and the populations, interventions, and outcomes that were the target of the NMA (Salanti et al ., 2014).”
Zhang, Y., Zhou, X., Yang, L., Hetrick, S. E., Weisz, J. R., Cuijpers, P., Barth, J., Del Giovane, C., Yuan, S., Cohen, D., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-020198 · Cited by 20
“In addition, we will evaluate the quality of evidence for primary outcome by using the Grading of Recommendations, Assessment, Development and Evaluations framework, which characterises the quality of a body of evidence on the basis of the study limitations, imprecision, inconsistency, indirectness and publication bia…”
MethodJeon, J., Park, S. H., Choi, J., Han, S. M., Kim, H., Shim, S. R., & Hyun, J. K. (2024) · Acta Biomaterialia · doi:10.1016/j.actbio.2024.06.011 · Cited by 24
Atypical antipsychotics for autism spectrum disorder: a network meta-analysis
Meza, N., Franco, J. V. A., Sguassero, Y., Núñez, V., Liquitay, C. M. E., Rees, R., Williams, K. J., Rojas, V., Rojas, F., Pringsheim, T. M., et al. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014965.pub2 · Cited by 41
Subramani, Y., Nagappa, M., Kumar, K., Fochesato, L.-A., Chohan, M. B. Y., Zhu, Y. F., Armstrong, K., & Singh, S. (2020) · BMC Anesthesiology · doi:10.1186/s12871-020-01116-5 · Cited by 25
Saglietto, A., Ballatore, A., Gaïta, F., Scaglione, M., Ponti, R. D., Ferrari, G. M. D., & Anselmino, M. (2021) · European Heart Journal - Quality of Care and Clinical Outcomes · doi:10.1093/ehjqcco/qcab066 · Cited by 22
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.