Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 801–900 of 999
Huang, Q., You, M. J., Huang, W., Chen, J., Zeng, Q., Jiang, L., Du, X., Liu, X., Hong, M., & Wang, J. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1050412 · Cited by 8
Zhou, J., Yang, M., Wang, F., Liu, S., Hei, M., & Jiang, M. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107601 · Cited by 9
Gao, X., Zhang, H., Fu, X., Yang, Y., & Dou, J. (2024) · Journal of Clinical Nursing · doi:10.1111/jocn.17136 · Cited by 12
Pessôa, R. L., Kessler, V. G., Becker, G. G., Garcia, G. M., Araldi, P. V. D., & Aver, P. V. (2024) · Vascular and Endovascular Surgery · doi:10.1177/15385744241253201 · Cited by 10
“19,20 This framework enables the classi fi cation of con fi dence levels in the results as high, moderate, low, or very low.”
Comparison of interventions for Barrett’s esophagus: A network meta-analysis
Zhang, Q., Li, M., Jin, X., Zhou, R., Ying, Y., Wu, X., Jing, J., & Pan, W. (2024) · PLoS ONE · doi:10.1371/journal.pone.0302204 · Cited by 7
Guo, J., Wei, M., Zhang, W., Jiang, Y., Li, A., Wang, C., Yin, D., Sun, A., & Gong, Y. (2025) · Frontiers in Pharmacology · doi:10.3389/fphar.2025.1517272 · Cited by 9
More data, more answers: picking the optimal antidepressant
Parikh, S. V., & Kennedy, S. H. (2018) · The Lancet · doi:10.1016/s0140-6736(18)30421-5 · Cited by 7
Johnston, B. W., Hill, R. A., Duarte, R. V., Chean, C. S., McAuley, D. F., Blackwood, B., Pace, N. L., & Welters, I. D. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1149-7 · Cited by 7
Wu, S., Zhou, Q., Zeng, X., Zhang, J.-X., Yang, Z., Yang, Q., Zhang, Z.-L., Chen, Y., Sun, F., & Zhan, S.-Y. (2021) · Chinese Medical Journal · doi:10.1097/cm9.0000000000001672 · Cited by 7
“Additionally, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework was used to assess the quality of evidence contributing to each network estimation, which characterizes the quality of a body of evidence on the basis of study limitations, imprecision, inconsistency, indirectness, a…”
MethodCurrent Status of Probiotics for Preterm Infants
Deshmukh, M., & Patole, S. (2021) · The Indian Journal of Pediatrics · doi:10.1007/s12098-021-03736-2 · Cited by 7
“, placebo or ‘standard treatment only’ as control), and the setup [21, 22].”
BackgroundRuszkowski, J., Majkutewicz, K., Rybka, E., Kutek, M., Dębska-Ślizień, A. M., & Witkowski, J. M. (2021) · Biomedicine & Pharmacotherapy · doi:10.1016/j.biopha.2021.112044 · Cited by 9
Nie, S., Zhou, S., & Huang, W. (2022) · PLoS ONE · doi:10.1371/journal.pone.0275097 · Cited by 8
“[28] which was based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Groups.”
MethodAdverse effects of immunotherapies for multiple sclerosis: a network meta-analysis
Tramacere, I., Benedetti, M. D., Capobussi, M., Castellini, G., Citterio, A., Giovane, C. D., Frau, S., González‐Lorenzo, M., Mantia, L. L., Moja, L., et al. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012186 · Cited by 7
Medley, N., Vogel, J. P., Care, A., & Alfirević, Ž. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012505 · Cited by 8
Antidepressants for major depression disorder in older people: a network meta-analysis
Robertson, L., Bertolini, F., Meader, N., Davies, S., Barbui, C., Gilbody, S. M., & Churchill, R. C. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013394 · Cited by 7
Zhong, C. C., Wong, C. H. L., Cheung, W. K. W., Yeoh, E., Hung, C. T., Yip, B. H. K., Wong, E. L., Wong, S. Y. S., & Chung, V. C. (2022) · International Journal of Integrated Care · doi:10.5334/ijic.6018 · Cited by 8
Ke, Y., Wang, J., Wang, W., Guo, S., Dai, M., Wu, L., Bao, Y., Li, B., Ju, J., Xu, H., et al. (2022) · International Journal of Cardiology · doi:10.1016/j.ijcard.2022.05.060 · Cited by 7
Fu, S., Zhang, K., Gu, M., Liu, Z., Sun, W., & Xiao, M. (2019) · Medicine · doi:10.1097/md.0000000000014709 · Cited by 6
Luo, Y., Chaimani, A., Furukawa, T. A., Kataoka, Y., Ogawa, Y., Cipriani, A., & Salanti, G. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1413 · Cited by 6
Self-citationDeclercq, I., Leontjevas, R., Verboon, P., VRIENDT, P. D., Gerritsen, D. L., & van Hooren, S. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05117-8 · Cited by 9
“Quality of evidence was rated using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) principles for NMA [44–46].”
MethodLin, J., Yang, K., Zhou, Q., Ye, Q., Chen, Z., Zhang, P., Zhou, M., & Pan, L. (2024) · International Archives of Allergy and Immunology · doi:10.1159/000542797 · Cited by 10
“Additionally, we employed the Grading of Recommendations Assessment, Development, and Evaluation framework (GRADE) to determine the certainty of evidence [17].”
Minozzi, S., Cinquini, M., Arienti, C., Battain, P. C., Brigadoi, G., Vicario, M. D., Domenico, G. D., Farma, T., Federico, S., Innocenti, T., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111811 · Cited by 7
Xie, J., Guo, J., & Wang, B. (2025) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2025.05.001 · Cited by 6
Morris, T., Nevill, J., Nevill, C. R., Bradbury, N. V., Freeman, S. C., Cooper, N. J., & Sutton, A. J. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111839 · Cited by 8
Delaigue, F., Wardani, H. M., Descamps, J., Ollivier, M. P., Nizard, R., & Bouché, P.-A. (2025) · Knee Surgery and Related Research · doi:10.1186/s43019-025-00278-1 · Cited by 6
“Finally, the validity of the meta-analysis was evaluated using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) framework [21].”
Xin, X., Guo, Y., Liu, L., Liu, Q., & Xie, J. (2025) · BMC Sports Science Medicine and Rehabilitation · doi:10.1186/s13102-025-01316-x · Cited by 10
“The overall risk of bias for each study was assessed comprehensively according to the standard [49], and the certainty of evidence for the main outcomes in the network meta-analysis was evaluated using the GRADE approach [50].”
Zhang, Q., Li, H., Chen, C., Li, M., Song, J., Pan, S., Shen, B., & Huang, Y. (2025) · Therapeutic Advances in Gastroenterology · doi:10.1177/17562848251399033 · Cited by 8
“26 To assess potential publication bias, comparison-adjusted funnel plots were constructed, and their symmetry was visually inspected to determine if studies with smaller sample sizes influenced the efficacy results.”
Zhang, L., Zhu, B., Zhang, R., Liu, H., Li, P., Liao, J., Shen, H., & Tian, L. (2022) · Medicine · doi:10.1097/md.0000000000031831 · Cited by 6
“Though the network meta-analysis methodology has become a critical component of evidence synthesis and decision-making in healthcare, there are potential drawbacks that may require consideration, such as the observational nature of indirect comparisons, assumptions that underlie the model and issue with inconsistency.…”
BackgroundAiolfi, A., Bona, D., Gambero, F., Sozzi, A., Bonitta, G., Rausa, E., Bruni, P. G., Cavalli, M., & Campanelli, G. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000250 · Cited by 6
“The con fi dence in estimates of the outcome was assessed using Con fi dence in Network Meta ‐ Analysis (CINeMA) [36] .”
Falci, S. G. M., Fernandes, I. A., Guimarães, M. T. B. Á., Galvão, E. L., de Souza, G. M., & Al‐Moraissi, E. A. (2024) · Clinical Oral Investigations · doi:10.1007/s00784-024-05625-2 · Cited by 6
“Treatments were ranked according to P-score (frequentist analogue of SUCRA) presenting the percentage of efficacy achieved by a therapy compared to an imaginary therapy considered the best without uncer - tainty [ 28 ].”
Osorio, D. L. S., Maldonado, D., Rijs, K., van der Marel, C., Klimek, M., & Calvache, J. A. (2024) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-024-02760-y · Cited by 9
“…for each outcome subjected to network meta-analysis using the grading of recommendations assessment, development, and evaluation system (GRADE) 19 by using the Confidence in Network Meta-Analysis methodology (CINeMA, Institute of Social and Preventive Medicine, University of Bern, Switzerland).”
InfluentialEffectiveness of safety-netting approaches for acutely ill children: a network meta-analysis
Burvenich, R., Bos, D. A. G., Lowie, L., Peeters, K., Toelen, J., Wynants, L., & Verbakel, J. Y. J. (2024) · British Journal of General Practice · doi:10.3399/bjgp.2024.0141 · Cited by 6
“The Confidence in Network Meta-Analysis (CINeMA) approach 25–27 and online platform (https://cinema.ispm.unibe. ch) were used to assess the certainty of evidence for all outcomes: within-study RoB, reporting bias, indirectness, imprecision (clinically important size defined as odds ratio [OR] >1.2…”
InfluentialGluz, O., Xu, B., Nanda, R., Dasgupta, A., Kaushik, A., Verret, W. J., Sharma, A., Singh, B., & Rugo, H. S. (2025) · Therapeutic Advances in Medical Oncology · doi:10.1177/17588359251320285 · Cited by 6
“18 Therefore, a feasibility assessment of the meta-analytic framework was conducted by evaluating the availability of evidence and heterogeneity of prognostic factors at trial and patient levels, as well as the approaches to meta-analyze the effect sizes of SG versus TPC from the TROPiCS-02 and…”
Fahrbach, K. R., Cichewicz, A. B., Chu, H., Fusco, M. D., Burnett, H. F., Volkman, H. R., Akin‐Fajiye, M., Mendoza, C. F., & Cappelleri, J. C. (2025) · Expert Review of Vaccines · doi:10.1080/14760584.2025.2505754 · Cited by 7
“Bayesian diagnosis of MCMC convergence was assessed using trace plots, the Gelman – Rubin statistic, and effective sample size estimates to ensure reliable posterior inference [55].”
Lee, C., Kang, Y., Ho, C., Lin, C. S., Chen, P., Wu, Y., & Huang, T. (2020) · Medicine · doi:10.1097/md.0000000000019618 · Cited by 7
“SUCRA is a simple transformation of mean rank, providing a hierarchy of treatments and accounts for location and variance of all relative treatment effects.([26,27]) The larger the SUCRA value (ie, closer to 1) is, the better the rank of the intervention.”
Chen, L., Xu, X., Cao, H., & Li, H. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-042997 · Cited by 6
Zhao, Y., Wang, Y., Shan, L., Peng, C., Zhang, W., & Zhao, X. (2021) · Scientific Reports · doi:10.1038/s41598-021-86102-8 · Cited by 7
Raja, V., Farajzadegan, Z., Mansourian, M., Ghasemi, K., Aboutalebi, M. S., Nouri, R., & Mokarian, F. (2022) · Disease Markers · doi:10.1155/2022/4119345 · Cited by 6
“I2 statistics was calculated to assess the heterogeneity by using the Markov chain Monte Carlo (MCMC), and I2 > 50% represented high heterogeneity among the same comparative groups [44, 45].”
MethodCortese, S., Giovane, C. D., Chamberlain, S. R., Philipsen, A., Young, S., Bilbow, A., & Cipriani, A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058102 · Cited by 6
“41 42 CINeMA is a software which uses the netmeta R-package for performing NMA of the data.”
Zhong, P.-Y., Ma, Y., Shang, Y.-S., Niu, Y., Bai, N., & Wang, Z.-L. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.899701 · Cited by 6
“Grades of Recommendations Assessment, Development and Evaluation (GRADE) was applied to evaluate the quality of each outcome according to direct, indirect, and network comparison, respectively (17, 18).”
MethodA Network Meta-Analysis of Intravenous Versus Oral Acetaminophen for Patent Ductus Arteriosus
Olowoyeye, A., Nnamdi-Nwosu, O., Manalastas, M., & Okwundu, C. I. (2022) · Pediatric Cardiology · doi:10.1007/s00246-022-03053-1 · Cited by 8
Watanabe, J., Kataoka, Y., Koike, A., Miki, A., Shiozawa, M., SAKURAGI, M., Harao, M., Kitayama, J., & Sata, N. (2023) · Breast Cancer · doi:10.1007/s12282-023-01460-7 · Cited by 6
Luo, J., Tao, Y., Xu, Y., Yang, L., Zhang, H., Chen, Y., Liu, X., Zhang, Y., Lan, R., Chen, Y., et al. (2023) · Journal of Diabetes and its Complications · doi:10.1016/j.jdiacomp.2023.108518 · Cited by 9
Li, L., Wang, A. T., Fang, Q., & Moosbrugger, M. E. (2023) · JMIR Research Protocols · doi:10.2196/40383 · Cited by 9
“The Grading of Recommendations Assessment, Development, and Evaluation will be used to assess the quality of the selected studies in the NMA and pairwise meta-analysis [63-65].”
MethodHao, S., Zhang, C., & Meng, X. (2023) · Medicine · doi:10.1097/md.0000000000035152 · Cited by 7
“[13] 3”
Stroke Risk After Mitral Valve Interventions: A Systematic Review and Network Meta-Analysis
Kazemian, S., Bansal, K., Kuno, T., Gupta, T., Goel, K., Khera, S., Alkhouli, M. A., & Kolte, D. S. (2024) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2024.05.030 · Cited by 6
Willis, M. S., Asseburg, C., & Neslusan, C. A. (2019) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2019.01.005 · Cited by 10
Bereznyakova, O., Dewar, B., Dowlatshahi, D. D., Howard, V. J., Hamel, C., Gocan, S., Fedyk, M., & Shamy, M. C. F. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032140 · Cited by 6
Afify, M. A., Ahmed, I. G. G., Alkahtani, T. A., Altulayhi, R. I., Alrowili, A. S. M., Ghozy, S. A., Bin-Jumah, M. N., & Abdel‐Daim, M. M. (2020) · Environmental Science and Pollution Research · doi:10.1007/s11356-020-11267-0 · Cited by 6
Pharmacological Interventions for Post-operative Sore Throat (POST): A Network Meta-analysis
(2020) · Signa Vitae · doi:10.22514/sv.2020.16.0085 · Cited by 6
“Whenever heterogeneity is present, the splitting of direct and indirect comparisons would be done to explore any possible sources [16].”
Zhao, X., Meng, L., & Chen, Y. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-040906 · Cited by 8
Interventions for treating people with symptoms of bladder pain syndrome: a network meta-analysis
Imamura, M., SCOTT, N. W., Ogah, J. A., Ford, A. A., Wallace, S. A., Dubos, Y. A., & Brazzelli, M. G. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013325 · Cited by 6
Naing, C. M., Poovorawan, Y., & Kew, S.-T. (2018) · BMC Infectious Diseases · doi:10.1186/s12879-018-3506-x · Cited by 8
Network meta-analyses in reproductive medicine: challenges and opportunities
Wang, R., Seidler, A. L., Askie, L. M., Norman, R. J., Bhattacharya, S., van Wely, M., & Mol, B. W. (2020) · Human Reproduction · doi:10.1093/humrep/deaa126 · Cited by 6
“In addition, it is also necessary to report the overall certainty (quality) of evidence in a network meta-analysis by incorporating risk of bias, imprecision, inconsistency, indirectness and publication bias into the assessment of each comparison (Puhan et al., 2014; Salanti et al., 2014).”
BackgroundCostantini, I., Paul, E., Caldwell, D. M., López-López, J. A., & Pearson, R. M. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01500-9 · Cited by 6
“propriate tool to summarise and assess the quality of the evidence across included studies [106].”
MethodSelf-citationReactive air surfaces for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013622 · Cited by 6
Methodological considerations in network meta-analysis
Hassan, S., Ravishankar, N., & Nair, N. S. (2015) · International Journal of Medical Science and Public Health · doi:10.5455/ijmsph.2015.21012015131 · Cited by 6
“This can be represented in a twodimensional treatment specific plot[26] in which the horizontal axis represents the possible ranks of all the treatments in NMA and the vertical axis has the probability for that particular treatment to assume each of the possible ranks with respect to a given specific outcome.”
BackgroundFocsa, L.-C., Bouché, P.-A., Apard, T., Querel, D., Rousseau, M.-A., & Descamps, J. (2025) · EFORT Open Reviews · doi:10.1530/eor-2024-0014 · Cited by 5
“The quality of evidence for the main outcomes was evaluated using the GRADE framework (45).”
Vo, A., Ta, K.-N. T., & Chuang, K.-J. (2025) · Asia-Pacific Journal of Oncology Nursing · doi:10.1016/j.apjon.2025.100671 · Cited by 9
“26,27”
Wu, R., Zhang, Y., Yuan, X., Yan, X., & Fu, X. (2025) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2025.1489382 · Cited by 9
“The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework was utilized to evaluate the certainty of the evidence contributing to the network estimates for both primary and secondary outcomes (63).”
Comparing the biopsy strategies of prostate cancer: a systematic review and network meta-analysis
Huang, Y., Wei, C., Chen, F., Zhang, Y., & Pu, F. (2025) · BMC Cancer · doi:10.1186/s12885-025-14203-y · Cited by 5
“In this approach, direct evidence from RCTs starts at high quality and can be downgraded based on risk of bias, indirectness, imprecision, inconsistency (or heterogeneity) and publication bias to levels of moderate, low and relatively low quality [28].”
Tan, L., Huang, D., Liu, B., Ossowski, Z. M., Wang, N., & Yan, W. (2025) · International Journal of Obesity · doi:10.1038/s41366-025-01777-7 · Cited by 10
“Given the diversity of the included studies, and in particular their differences in instrumentation and units used to measure in fl ammatory markers, the SMD was speci fi cally chosen as the measure in this study to accurately assess the effect sizes of CRP, IL-6, leptin, and adiponectin.”
Martino, E. A., Caridà, G., Lofaro, D., Bruzzese, A., Labanca, C., Mendicino, F., Lucia, E., Olivito, V., Puccio, N., Neri, A., et al. (2025) · ESMO Open · doi:10.1016/j.esmoop.2025.105514 · Cited by 7
Wang, H., Wang, H., Zhan, E., & Liu, X. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1653449 · Cited by 5
Du, M., Chen, L., Xia, L., Li, Y., Ma, E., Hu, Z., & Gao, X. (2025) · BMC Sports Science Medicine and Rehabilitation · doi:10.1186/s13102-025-01267-3 · Cited by 9
“Ultimately, the quality of evidence was categorized into four levels: high, moderate, low, and very low [29].”
Trinquet, N. M., Kellow, N. J., Eviana, R., Williamson, G., & Dordevic, A. L. (2025) · American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2025.09.020 · Cited by 5
Network meta-analysis of treatment strategies for calcified coronary lesions
Maremmani, M., Gasparini, G. L., Bruno, F., Berti, S. L., Digiacomo, S., Ebrahimi, R., Templin, C., Canova, P. A., Pesarini, G., & Vece, D. D. (2025) · Cardiovascular Intervention and Therapeutics · doi:10.1007/s12928-025-01214-6 · Cited by 5
Vasireddy, S., Biswas, S. K., Kollu, R., Krishnan, E., Khan, M. S., Dr.Mohammed Fasil, C., Jayakumar, A. A., & Acharya, R. (2025) · BMC Neurology · doi:10.1186/s12883-025-04490-0 · Cited by 7
“We assessed certainty using GRADE for network meta-analysis, [21] considering risk of bias, inconsistency, indi-rectness, imprecision, and publication bias.”
Leong, Z. N., Mak, I. E. K., Chen, Y., Du, X., & Kim, J. E. (2026) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2026.105725 · Cited by 5
Zhang, Y., Li, D., Tan, Z., & Chen, X. (2026) · Maturitas · doi:10.1016/j.maturitas.2026.108972 · Cited by 5
Tan, X., Li, Y., Xi, J., Guo, S., Su, H., Chen, X., & Liang, X. (2021) · Medicine · doi:10.1097/md.0000000000027266 · Cited by 6
“In a complex setting of different treatment choices, including several optional interventions and some therapeutic strategies that have not been directly compared, a network meta-analysis can provide direct and indirect comparisons of various treatment strategies simultaneouslywithin a single network and rank the opti…”
MethodYang, H., Xiao, Y.-Q., Liu, J., Xu, G., Li, J., Xiao, Z.-Y., Zhou, J., Zheng, X., Liu, L.-Y., Yu, Z., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059090 · Cited by 7
Hong, B., Zheng, J., Chen, R., Zheng, C., Du, B., Ni, R., & Yang, J. (2023) · Drug Safety · doi:10.1007/s40264-023-01357-6 · Cited by 5
“Publication bias was assessed by comparing funnel plots visually [20, 21].”
MethodNaughton, A., Ryan, É. J., Keenan, R. A., Thomas, A. Z., Smyth, L. G., Manecksha, R. P., Flynn, R. J., & Casey, R. G. (2023) · Journal of Endourology · doi:10.1089/end.2023.0107 · Cited by 5
Wei, S., He, Y., Liu, K., Wang, R., & Wang, Y. (2024) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2024.105558 · Cited by 6
Favre-Bulle, E., Nyfeler, S., Schorderet, C., Risso, G., Bassolino, M., & Sattelmayer, M. (2024) · PeerJ · doi:10.7717/peerj.17799 · Cited by 14
“The transitivity assumption was examined for all studies included in the network meta-analysis ( Salanti et al., 2014 ).”
Lu, F., Qin, S., Liu, C. X., Chen, X., Dai, Z., & Li, C. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1370481 · Cited by 5
Acupuncture for heart failure: a Bayesian network systematic review and meta-analysis protocol
Zhu, F., Yin, S., Zheng, G., Zhong, Y., Gan, D., Deng, L., Chen, S., Yang, L., & Li, L. (2021) · Annals of Palliative Medicine · doi:10.21037/apm-21-2131 · Cited by 5
“Network metaanalysis (NMA) assesses both direct and indirect evidence (20-22), so we aim perform NMA to comprehensively evaluate the efficacy and safety of different acupuncture methods for HF.”
BackgroundPertl, M., Pérez, S., Collier, S., Guinan, E. M., Monahan, G., Verling, K., Wallace, E., Walsh, A., & Doyle, F. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02107-y · Cited by 8
“In line with previous NMAs [25, 28, 35], we will extract pertinent data to explain clustering from cluster randomised trials and extract only data relating to the first study period from cross-over trials to prevent carry-over effects.”
BackgroundMethodInfluentialCho, S.-H., & Shin, I. (2022) · Healthcare · doi:10.3390/healthcare10122371 · Cited by 5
“The GRADE approach was used to assess the quality of the evidence behind the ranking of treatments from NMAs [37] (item 12).”
MethodJunior, L. C. S., dos Santos, J. R., Reis, A., Faria‐e‐Silva, A. L., & Leal, P. C. (2023) · Clinical Oral Investigations · doi:10.1007/s00784-023-04981-9 · Cited by 5
Guo, X., Lin, Y., He, F., Jin, Y., Chen, S., Li, T., Wu, C., Zhang, L., & Chen, X. (2023) · Expert Review of Anti-infective Therapy · doi:10.1080/14787210.2023.2238899 · Cited by 6
Valbenazine for tardive dyskinesia: a systematic review and network meta-analysis
Kishi, T., Sakuma, K., & Iwata, N. (2023) · International Clinical Psychopharmacology · doi:10.1097/yic.0000000000000485 · Cited by 5
“…meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach (Salanti et al ., 2014; Nikolakopoulou et al ., 2020; Papakonstantinou et al ., 2020).”
Efficacy of immune nutrients in severe acute pancreatitis: A network meta-analysis
Tao, X., Yang, Y., Xu, S., & Xiong, Q. (2023) · Medicine · doi:10.1097/md.0000000000035615 · Cited by 8
“[20,21]”
Kishi, T., Sakuma, K., Saito, T., Nakagawa, A., Kato, M., & Iwata, N. (2024) · Neuropsychopharmacology Reports · doi:10.1002/npr2.12414 · Cited by 7
Hong, B., Du, B., Chen, R., Zheng, C., Ni, R., Liu, M., & Yang, J. (2024) · BMC Medicine · doi:10.1186/s12916-024-03285-3 · Cited by 5
“To rule out small studies that produced different results, we explored the possibility of publication bias by visually comparing the adjusted funnel plots and performing Egger’s test [22, 23].”
MethodYamasaki, A., Tomita, K., Inui, G., Okazaki, R., & Harada, T. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e31186 · Cited by 6
“Statistical heterogeneity (I 2 ) is a statistic that indicates the percentage of variance in a pairwise meta-analysis [36,37].”
BackgroundComparison of Anesthetics for Laryngeal Mask Airway Insertion: A Network Meta-Analysis
Goertzen, C., Goertzen, E., Zanjir, M., Dare, C., Azarpazhooh, A., & Wong, M. (2024) · Anesthesia Progress · doi:10.2344/22-00033 · Cited by 5
Arora, N. K., Donath, L., Owen, P. J., Miller, C. T., Kaczorowski, S., Saueressig, T., Pedder, H., Mundell, N. L., Tagliaferri, S. D., Diwan, A. D., et al. (2024) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2024-002108 · Cited by 5
“The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach 62 will be used by two independent assessors to assess the certainty of evidence from the NMA at discrete time points results.”
MethodMedley, N., Donegan, S., Nevitt, S. J., Goodfellow, L., Hampson, L., Caldwell, D. M., Smith, C. T., & Alfirević, Ž. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013455 · Cited by 5
Self-citationNikolopoulos, G. K., Yiallourou, A., Argyriou, C., Bonovas, S., Georgiadis, G. S., & Lazarides, M. K. (2019) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2019.06.495 · Cited by 5
Bighelli, I., Rodolico, A., Pitschel‐Walz, G., Hansen, W., Barbui, C., Furukawa, T. A., Salanti, G., & Leucht, S. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-035073 · Cited by 9
“28 We will employ the ‘separating indirect from direct evidence approach’, which separates direct evidence from indirect evidence and then evaluates their agreement to evaluate local consistency.”
MethodSelf-citationA network meta-analysis of maintenance therapy in chronic lymphocytic leukemia
Lee, C., Chen, P., Lin, C. S., Wang, C., & Ho, C. (2020) · PLoS ONE · doi:10.1371/journal.pone.0226879 · Cited by 5
Ding, W., Tan, Y., Qian, Y. N., Xue, W., Wang, Y., Jiang, P., & Xu, X. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229492 · Cited by 5
“The transitivity was assessed by examining the patient baseline characteristics across studies (age, gender, performance status and tumor stage), treatment stage and treatment protocol [16].”
MethodTreatments for alopecia areata: a network meta-analysis
Candia, M. P. N., Sánchez, G., Arévalo-Rodríguez, I., Vañó-Galván, S., Rodriguez, M. G. U., Giraldo, L., Saceda‐Corralo, D., Tobías, A., Tomlinson, J., Ortega-Quijano, D., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013719 · Cited by 5
Zitikyte, G., Roy, D. C., Dhaliwal, S., Lun, R., Hutton, B., Shorr, R., & Dowlatshahi, D. D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250553 · Cited by 5
“Transitivity will be attenuated by only including studies for which methodology and characteristics are as similar as possible [22].”
BackgroundJhou, H., Ou‐Yang, L., Lin, M., Chen, P., Ho, C., & Lee, C. (2021) · Postgraduate Medical Journal · doi:10.1136/postgradmedj-2021-140076 · Cited by 6
Rokkas, T., Ekmektzoglou, K. A., & Niv, Y. (2021) · Journal of Gastrointestinal and Liver Diseases · doi:10.15403/jgld-3804 · Cited by 6
“the agreement between direct and indirect evidence and the transitivity assumption, as these are critical when conducting an NWM [22, 23].”
BackgroundCharitakis, E., Metelli, S., Karlsson, L., Antoniadis, A. P., Liuba, I., Almroth, H., Jönsson, A. H., Schwieler, J. H., Sideris, S. K., Tsartsalis, D., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12020433 · Cited by 6
“ch; last accessed: 2 December 2021) [23] (More details concerning the methodology of the trial can be found in Sections S5–S13 in the Supplement).”
MethodSelf-citationKishi, T., Sakuma, K., & Iwata, N. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-02286-1 · Cited by 5
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.