Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 601–700 of 999
An appraisal of meta‐analysis guidelines: how do they relate to safety outcomes?
Bennetts, M., Whalen, E., Ahadieh, S., & Cappelleri, J. C. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1219 · Cited by 17
“An approach has been proposed to evaluate the quality of evidence from an NMA based on the GRADE working group methods for pairwise MA (Salanti et al., 2014).”
MethodAlternative reactive support surfaces (non-foam and non-air-filled) for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013623.pub2 · Cited by 17
Subramani, Y., Nagappa, M., Kumar, K., Mortuza, R., Fochesato, L.-A., Chohan, M. B. Y., Martin, J. E., Armstrong, K., & Singh, S. I. (2020) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2020.110102 · Cited by 13
Bettiol, A., Avagliano, L., Lombardi, N., Crescioli, G., Emmi, G., Urban, M. L., Virgili, G., Ravaldi, C., & Vannacci, A. (2021) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.2164 · Cited by 21
Ranking of treatments in network meta-analysis: incorporating minimally important differences
Curteis, T., Wigle, A., Michaels, C. J., & Nikolakopoulou, A. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02499-0 · Cited by 22
“…Monte Carlo simulation, and ranking metrics are then generated by calculating aggregate statistics (typically the median, rather than the mean being a robust measure of central tendency that accounts for uncertainty and asymmetry in posterior distributions) over all Monte Carlo simulations [11].”
Updating the PRISMA reporting guideline for network meta-analysis: a scoping review
Veroniki, A. A., Tricco, A. C., Rangira, D., McKenzie, J. E., Li, T., Straus, S. E., Smith, M., Catalá-López, F., Wolfe, D. M., Nincic, V., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111985 · Cited by 29
Diniz, P., Pacheco, J., Fernandes, R. M., Pereira, H., Ferreira, F. C., & Kerkhoffs, G. M. M. J. (2021) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-021-06613-9 · Cited by 14
Wu, H., Zhang, S., Lin, X., Wang, S., & Zhou, P. (2021) · Reproductive Biology and Endocrinology · doi:10.1186/s12958-021-00782-5 · Cited by 14
Zhao, X., Huang, X., Cai, Y., Cao, T., & Wan, Q. (2022) · Aging & Mental Health · doi:10.1080/13607863.2022.2026879 · Cited by 14
Rokkas, T., & Hold, G. L. (2023) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002519 · Cited by 15
Riva, C. G., Kelly, M. E., Vitellaro, M., Rottoli, M., Aiolfi, A., Ferrari, D., Bonitta, G., & Rausa, E. (2023) · Techniques in Coloproctology · doi:10.1007/s10151-023-02868-1 · Cited by 15
“The confidence in estimates of the outcome was assessed using Confidence in Network Meta-Analysis (CINeMA) [24].”
MethodWen, L., Xie, L., Gong, F., Zhang, S., & Xi, T. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1277284 · Cited by 15
Rokkas, T., Gisbert, J. P., Ekmektzoglou, K. A., Dassopoulos, T., Niv, Y., & O’Morain, C. A. (2024) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002751 · Cited by 15
“In addition to heterogeneity, we assessed consistency, as this is critical when conducting an NWM [24,25,26].”
MethodHu, W., Seah, V., Huang, V., & Kim, J. E. (2024) · Advances in Nutrition · doi:10.1016/j.advnut.2024.100216 · Cited by 16
“The quality of evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation approach, considering study limitations, inconsistency, indirectness, imprecision, and publication bias for all outcomes [22,23].”
MethodSun, L., Dai, X., Zhu, S., Liu, Z., & Zhongming, Z. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.02.092 · Cited by 18
Asha, M. Z., Al-Asaad, Y., & Khalil, S. F. H. (2021) · IBRO Neuroscience Reports · doi:10.1016/j.ibneur.2021.08.003 · Cited by 14
Bronchoscopic lung volume reduction procedures for emphysema
Xu, W., Wang, J., He, X., Wang, J., Wu, D., & Li, G. (2020) · Medicine · doi:10.1097/md.0000000000018936 · Cited by 14
“Additionally, we assessed the quality of evidence contributing to each network estimate using the GRADE framework, which characterizes the quality of a body of evidence on the basis of the study limitations, imprecision, inconsistency, indirectness, and publication bias for the primary outcomes.([16]) To assess the in…”
MethodFoam surfaces for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013621.pub2 · Cited by 16
Guo, Q., Yan, F., Ouyang, P., Xie, Z., Wang, H., Yang, W., & Pan, X. (2021) · Journal of Cardiovascular Electrophysiology · doi:10.1111/jce.15127 · Cited by 15
“We assessed the overall quality of evidence for every comparison and ranking on each of the outcomes based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework15 and the method introduced by Salanti et al.16 The method allows for the quality assessment of a specific comparison bas…”
MethodWhich Is the Best Biologic for Nasal Polyps: An Updated Network Meta‐Analysis
Xi Tao, X. U., Lin, J., Luo, M., & Wu, Q. (2025) · Clinical and Translational Allergy · doi:10.1002/clt2.70114 · Cited by 14
Seppelt, P. C., Mas‐Peiró, S., Rosa, R. D., Dimitriasis, Z., Zeiher, A. M., & Vasa‐Nicotera, M. (2020) · Clinical Research in Cardiology · doi:10.1007/s00392-020-01775-x · Cited by 16
Wang, J., Carru, C., Sedda, S., Fiori, P. L., Li, Z., & Chen, Z. (2023) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.15091 · Cited by 26
Zhang, M., Liang, Z., Li, Y., Meng, J., Jiang, X., Xu, B., Li, H., & Liu, T. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1234017 · Cited by 22
“The heterogeneity of each network was assessed by statistics τ 2 and I 2 , whose consistency (between direct and indirect evidence) was assessed using both global (Q statistics) and local methods (identifying inconsistent “hot spots” using the “node-split” function) (27, 35).”
MethodQi, S., Luo, X., Liu, S., Ling, B., Si, M., & Jin, H. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-074511 · Cited by 18
“15 We also used the Confidence in Network Meta-Analysis (CINeMA) 16 web application, which evaluates six domains (within-study bias, publication bias, indirectness, imprecision, heterogeneity and incoherence) and classifies the credibility of evidence into four grades: high, moderate, low and very…”
MethodNetwork meta-analysis: The way forward for evidence-based decisions
Jaiswal, N., & Field, R. (2024) · Clinical Epidemiology and Global Health · doi:10.1016/j.cegh.2024.101531 · Cited by 14
Li, W., Wu, C., Feng, W., Zhan, Y., Yang, L., Jia, H.-J., Pei, J., & Li, K. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000002006 · Cited by 14
Miura, T., Yamamoto, N., Shiroshita, A., Tsuge, T., Saitsu, A., Yoshitani, J., Nakao, S., & Takami, K. (2024) · JBJS Reviews · doi:10.2106/jbjs.rvw.24.00120 · Cited by 13
Afflitto, G. G., Aiello, F., Surico, P. L., Malek, D. A., Mori, T., Swaminathan, S. S., Maurino, V., & Nucci, C. (2025) · Ophthalmology · doi:10.1016/j.ophtha.2025.02.010 · Cited by 18
Systematic Reviews and Cancer Research: a Suggested Stepwise Approach
Kelley, G. A., & Kelley, K. S. (2018) · BMC Cancer · doi:10.1186/s12885-018-4163-6 · Cited by 13
“For SRPSR that include or focus on network meta-analyses [28], an alternative instrument based on the GRADE methodology has also been developed [29].”
MethodMidtiby, S. L., Wedderkopp, N., Larsen, R. T., Carlsen, A.-M. F., Mavridis, D., & Shrier, I. (2018) · Chiropractic & Manual Therapies · doi:10.1186/s12998-018-0209-8 · Cited by 16
Deng, L., Qiu, S., Wang, L., Li, Y., Wang, D., & Liu, M. (2019) · World Neurosurgery · doi:10.1016/j.wneu.2019.02.011 · Cited by 13
Sargeant, J. M., Bergevin, M. D., Churchill, K. J., Dawkins, K., Deb, B., Dunn, J. N., Hu, D., Logue, C. M., Meadows, S., Moody, C. M., et al. (2019) · Animal Health Research Reviews · doi:10.1017/s1466252319000227 · Cited by 15
“In a GRADE assessment for NMA, indirectness refers to the differences between the populations, interventions, and outcomes in the included studies and the populations, interventions, and outcomes that were the target of the NMA (Salanti et al., 2014).”
BackgroundMethodSharma, G., Pareek, T., Tyagi, S., Kaundal, P., Yadav, A. K., Thummala, Y., & Devana, S. K. (2021) · Scientific Reports · doi:10.1038/s41598-021-91364-3 · Cited by 18
Bian, Z., Yu, J., Tu, M., Liao, B., Huang, J., Izumoji, G., Sun, R., Xu, Y., Jiang, Y., He, X., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-056632 · Cited by 14
Zhao, F., Chen, L., Jiang, Y., Guo, Y., Lu, L., Lu, C.-L., Xue, X., Liu, X., Jin, X., Liu, J., et al. (2023) · Journal of Functional Foods · doi:10.1016/j.jff.2023.105508 · Cited by 13
Crescenzo, F. D., Foti, F., Ciabattini, M., Giovane, C. D., Watanabe, N., Schepisi, M. S., Quested, D. J., Cipriani, A., Barbui, C., & Amato, L. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012364 · Cited by 14
Zhou, X., Cipriani, A., Furukawa, T. A., Cuijpers, P. J., Zhang, Y., Hetrick, S. E., Pu, J., Yuan, S., Giovane, C. D., & Xie, P. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-018357 · Cited by 17
Bighelli, I., Salanti, G., Reitmeir, C., Wallis, S., Barbui, C., Furukawa, T. A., & Leucht, S. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019280 · Cited by 20
Self-citationComparative diagnostic accuracy of EUS needles in solid pancreatic masses: a network meta-analysis
Han, S. S., Bhullar, F. A., Alaber, O. A., Kamal, A., Hopson, P. E., Kanthasamy, K. A., Coughlin, S., Archibugi, L., Thiruvengadam, N. R., Moreau, C., et al. (2021) · Endoscopy International Open · doi:10.1055/a-1521-0166 · Cited by 13
Reporting of Cochrane systematic review protocols with network meta‐analyses—A scoping review
Wang, R., Dwan, K. M., Showell, M. G., van Wely, M., Mol, B. W., Askie, L. M., & Seidler, A. L. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1531 · Cited by 13
“Pairwise MA specific methods 23 (51) 11 (40) 12 (71)”
Gao, L., Yu, S., Cipriani, A., Wu, S., Huang, Y., Zhang, Z., Yang, J., Sun, Y., Yang, Z., Chai, S., et al. (2019) · Aging and Disease · doi:10.14336/ad.2019.0303 · Cited by 13
“In addition, we used the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the quality of each mixed comparison and the total ranking of treatments [21, 22].”
MethodTang, Y., Tong, Q., Dai, N., & Xu, C. (2023) · Virology Journal · doi:10.1186/s12985-023-02001-6 · Cited by 13
“Additionally, we assessed the certainty of evidence contributing to network estimates of the primary outcomes with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework [20].”
MethodHerranz‐Gómez, A., Suso‐Martí, L., Varangot‐Reille, C., Barrachina-Gauchia, L., Casaña, J., López, L., Calatayud, J., & Cuenca‐Martínez, F. (2023) · Physical Therapy · doi:10.1093/ptj/pzad132 · Cited by 14
Zarrintan, A., Musmar, B., Ghozy, S. A., Mansour, M., Kadirvel, R., & Kallmes, D. F. (2024) · European Journal of Radiology · doi:10.1016/j.ejrad.2024.111506 · Cited by 12
Muñoz‐Pardeza, J., López‐Gil, J. F., Huerta‐Uribe, N., Hormazábal‐Aguayo, I., Izquierdo, M., & García‐Hermoso, A. (2024) · Cardiovascular Diabetology · doi:10.1186/s12933-024-02301-3 · Cited by 15
“In the final report, the interrelation between domains was considered, avoiding isolated judgments (e.g., substantial "heterogeneity" may impact increased "imprecision" in the relative treatment effects) [40].”
BackgroundHan, X., Zou, Y., & Zhang, S. (2024) · Sleep Medicine · doi:10.1016/j.sleep.2024.08.008 · Cited by 13
Tian, S., Liang, Z., Tian, M., Qiu, F., Yu, Y., Mou, H., & Gao, Y. (2024) · Age and Ageing · doi:10.1093/ageing/afae211 · Cited by 20
Bo, Y., Zhao, J., Liu, C., & Yu, T. (2025) · BMC Women s Health · doi:10.1186/s12905-025-03594-6 · Cited by 20
Zang, X., Tong, L. X., Ma, J., Zhang, Y., Zhang, B., Huang, Y., Zhou, D., Ding, L., Zhang, L., & Zhao, L. (2025) · Diabetes/Metabolism Research and Reviews · doi:10.1002/dmrr.70033 · Cited by 14
Celia, A. I., Vescovo, G. M., Sarto, G., Alessandri, C., Iaconelli, A., D’Amario, D., Frati, G., Conti, F., Sciarretta, S., Angiolillo, D. J., et al. (2025) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2025.152741 · Cited by 16
Huang, W., Liao, X., Tian, J., Wu, J., Shan, Y., & Zhou, W. (2018) · Medicine · doi:10.1097/md.0000000000013840 · Cited by 12
Thom, H. H. Z., Jansen, J. P., Shafrin, J. T., Zhao, L. M., Joseph, G., Cheng, H., Gupta, S., & Shah, N. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034147 · Cited by 13
Shaman, A. M., Smyth, B. J., Arnott, C. G., Palmer, S. C., Mihailidou, A. S., Jardine, M., Gallagher, M., Perkovic, V., & Jun, M. (2020) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.12201019 · Cited by 14
Treatment for gastrointestinal and pancreatic neuroendocrine tumours: a network meta-analysis
Walter, M. A., Nesti, C., Spanjol, M., Kollár, A., Bütikofer, L., Gloy, V., Dumont, R. A., Seiler, C. A., Christ, E., Radojewski, P., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013700.pub2 · Cited by 15
Niu, L., Xiao, L., Zhang, X., Liu, X., Liu, X., Huang, X., & Zhang, M. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.743486 · Cited by 16
“After that, publication bias were reflected by funnel plots (Chaimani et al., 2013; Salanti et al., 2014).”
MethodZhang, Q., Han, Y., Sun, S., Zhang, C., Liu, H., Wang, B., & Wei, S. (2022) · BMC Cardiovascular Disorders · doi:10.1186/s12872-022-02493-0 · Cited by 17
“Finally, the results were incorporated into the CINeMA application to assess the credibility of the results from each NMA [20].”
MethodPark, H., Huh, J.-Y., Lee, J. S., Lee, J. S., Oh, Y., & Lee, S. W. (2022) · Scientific Reports · doi:10.1038/s41598-022-09941-z · Cited by 13
Sun, W., Li, J., Yan, X., Liao, L., Li, S., Wang, X., Xiao, C., Shang, M., Chao, G., & Zhou, J. H. (2022) · Journal of Integrative and Complementary Medicine · doi:10.1089/jicm.2021.0392 · Cited by 12
Tokavanich, N., Prasitlumkum, N., Kewcharoen, J., Chokesuwattanaskul, R., Phannajit, J., Cheungpasitporn, W., AKOUM, N. W., Ranjan, R., Bunch, T. J., & Navaravong, L. (2023) · Journal of Cardiovascular Electrophysiology · doi:10.1111/jce.15831 · Cited by 12
“The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework was performed to assess if the certainty of information accounted for the network estimates of the main outcomes from individual studies.(20) The quality of treatment effects estimation was rated according to a GRADE working group…”
MethodYang, T. X., Tan, A. Y., Leung, W. H., Chong, D., & Chow, Y. F. (2023) · Cureus · doi:10.7759/cureus.38238 · Cited by 20
“To support the transitivity assumption in NMA, similar studies were selected [13].”
MethodHendra, F. N., Helder, M. N., Ruslin, M., Cann, E. M. V., & Forouzanfar, T. (2023) · Scientific Reports · doi:10.1038/s41598-023-32190-7 · Cited by 18
Effective luting agents for glass-fiber posts: A network meta-analysis
Angnanon, W., Thammajaruk, P., & Guazzato, M. (2023) · Dental Materials · doi:10.1016/j.dental.2023.10.015 · Cited by 15
Jin, K., Deng, L., Qiu, S., Tu, X., Li, J., Bao, Y., Yang, L., & Wei, Q. (2018) · Medicine · doi:10.1097/md.0000000000013342 · Cited by 12
Gao, C., Chen, G., Yang, H. J., Hua, Z., Xu, P., Wong, M. S., & He, C. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025971 · Cited by 15
“Assessment of quality of evidence Two authors will independently assess the quality of evidence using the Grading of Recommendations Assessment, Development and Evaluation instrument for NMA.(39) The overall strength of evidence for each outcome will be rated as very low, low, moderate and high quality.”
MethodInfluentialAthanasiou, A., Veroniki, A. A., Efthimiou, O., Kalliala, I. E. J., Naci, H., Bowden, S. J., Paraskevaidi, M., Martin‐Hirsch, P. L., Bennett, P. R., Paraskevaidis, E. A., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028009 · Cited by 15
“evidence).(61) Each network summary estimate will initially be judged as high quality, but it will be downgraded if this is judged appropriate according to the six domains.”
BackgroundSelf-citationLewis, K. A., Chaudhuri, D., Guyatt, G. H., Burns, K. E. A., Bosma, K. J., Ge, L., Karachi, T., Piraino, T., Fernando, S. M., Ranganath, N., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030407 · Cited by 13
“Quality of evidence We will use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess the certainty of evidence for each comparison.(34) Our certainty assessment will address the domains of RoB, imprecision, inconsistency (heterogeneity in estimates of effect across studies),…”
MethodDai, L., Jiang, Y., Wang, P., & Chen, K. (2021) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2021/2570472 · Cited by 15
“However, several limitations in this NMA should be noted.”
MethodInfluentialKelley, G. A., Kelley, K. S., & Pate, R. R. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-019512 · Cited by 12
Fountain, D. M., Jenkinson, M. D., Bryant, A., Vale, L. D., Bulbeck, H., Hart, M. G., & Barone, D. G. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013630 · Cited by 13
Oh, S., Purja, S., Shin, H., Kim, M. S., Park, S., Kronbichler, A., Smith, L., Eisenhut, M., Shin, J. I., & Kim, E. Y. (2022) · Vaccines · doi:10.3390/vaccines10101572 · Cited by 12
“pairwise analyses [22] and can rank treatments to inform clinical decisions [23].”
BackgroundDiarrhea, Constipation, and Irritable Bowel Syndrome
Wilhelm, S. M., McConachie, S. M., & Kale-Pradhan, P. B. (2017) · The American Pharmacists Association eBooks · doi:10.21019/pharmacotherapyfirst.ibs_overview · Cited by 12
Beds, overlays and mattresses for treating pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., Jammali‐Blasi, A., Ramsden, V., & McInnes, E. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013624.pub2 · Cited by 13
Fang, C., Tu, Y., Chang, S., Kuo, C., Fang, C., & Chou, F. (2023) · Nursing in Critical Care · doi:10.1111/nicc.12883 · Cited by 18
Treatment of diabetic kidney disease. A network meta-analysis
Büttner, F., Barbosa, C. V., Lang, H., Tian, Z., Melk, A., & Schmidt, B. M. W. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293183 · Cited by 14
Hesapçıoğlu, S. T., Kaşak, M., Aburşu, H., Kafalı, S., Ceylan, M. F., & Akyol, M. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.01.067 · Cited by 15
Imai, E., Kataoka, Y., Watanabe, J., Okano, H., Namekawa, M., Owada, G., Matsui, Y., & Yokozuka, M. (2024) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2024.01.043 · Cited by 12
Meng, Y., Xiang, S., Qu, L., & Li, Y. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1361186 · Cited by 18
Xie, Y., Zhu, X., Wang, L., Wan, Z., Yang, J., Su, C., Duan, S., Xu, C., & Kan, B. (2024) · BMC Psychiatry · doi:10.1186/s12888-024-05924-8 · Cited by 53
“This web application evaluates the confidence of findings for NMA [25].”
MethodXu, J., Pan, L., Wu, D., Yao, L., Jiang, W., Min, J., Xu, S., & Deng, Z. (2024) · BMC Cancer · doi:10.1186/s12885-024-12528-8 · Cited by 12
“The heterogeneity was assessed using the I 2 statistic, calculated from the MCMC samples, considering values above 50% indicative of considerable heterogeneity among the cohorts compared [15].”
MethodAbdollahi, S., Soltani, S., Ramezani‐Jolfaie, N., Mohammadi, M., Sherafatmanesh, S., Lorzadeh, E., & Salehi‐Abargouei, A. (2024) · BMC Nutrition · doi:10.1186/s40795-024-00907-0 · Cited by 12
Gao, K., Tao, J., Liang, G., Gong, C., Wang, L., & Wang, Y. (2025) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-025-05682-7 · Cited by 16
“The GRADE (Grading of Recommendations Assessment, Development, and Evaluation) framework was used to assess the quality of evidence [45], with ratings ranging from high (best) to very low (worst), and to assist in drawing conclusions from an NMA using a minimally contextualized framework [46].”
Kibret, K. T., Peeters, A., Tegegne, T. K., Mesfin, Y. M., & Nichols, M. S. (2025) · Current Nutrition Reports · doi:10.1007/s13668-025-00684-7 · Cited by 18
Deng, J., Abbas, U., Chang, O., Dhivagaran, T., Sanger, S., & Bozzo, A. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034741 · Cited by 11
Ranasinha, S., Hill, B., Teede, H. J., Enticott, J., Wang, R., & Harrison, C. L. (2021) · Obesity Reviews · doi:10.1111/obr.13406 · Cited by 14
“To evaluate publication bias, a comparison adjusted funnel plot was created for each direct pairwise comparison and Egger's linear regression test for funnel plot asymmetry was conducted to investigate small study effects.(15) Heterogeneity was assessed using the I(2) statistic.”
MethodLiang, Z., Xu, X., Chen, X., Zhuang, Y., Wang, R., & Chen, C. (2022) · Orthopaedic Surgery · doi:10.1111/os.13269 · Cited by 11
Comparative effectiveness of different acupuncture therapies for neck pain
Jo, H.-R., Noh, E.-J., Oh, S.-H., Choi, S.-K., Sung, W.-S., Choi, S.-J., Kim, D., Hong, S.-U., & Kim, E. (2022) · Medicine · doi:10.1097/md.0000000000029656 · Cited by 11
“Based on the assessment of each study limitation, inconsistency, indirectness, imprecision, and publication bias, the quality of the evidence can be maintained or downgraded to moderate, low, or very low quality.[21] Given that the quality of evidence might differ across paired comparisons, a GRADE approach previously…”
MethodLang, C. I., Dahmen, A., Vasudevan, P., Lemcke, H., Gäbel, R., Öner, A. Ö., İnce, H., David, R., & Wolfien, M. (2023) · Cytotherapy · doi:10.1016/j.jcyt.2023.01.013 · Cited by 12
Murdoch, I. A., Carver, A., Sultan, P., O’Carroll, J., Blake, L., Carvalho, B., Onwochei, D. N., & Desai, N. (2023) · Korean Journal of Anesthesiology · doi:10.4097/kja.23014 · Cited by 11
“The quality of evidence for every outcome was evaluated by two authors (IM and ND) using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system [23] and with the CINeMA software ® (Institute of Social and Preven-tative Medicine, University of Bern, Switzerland).”
Efficacy of electrophysical agents in fibromyalgia: A systematic review and network meta-analysis
Gikaro, J. M., Bigambo, F. M., Minde, V., & Swai, E. A. (2023) · Clinical Rehabilitation · doi:10.1177/02692155231170450 · Cited by 13
“We used the Grading of Recommendations Assessment, Development and Evaluation approach to assess the quality of the evidence behind the ranking of treatments from our network meta-analysis.(34) We conducted a network meta-analysis using the frequentist model according to the current Preferred Reporting Items for Syste…”
MethodTo Bev or Not to Bev during Ovarian Cancer Maintenance Therapy?
Sznurkowski, J. J. (2023) · Cancers · doi:10.3390/cancers15112980 · Cited by 13
“Notably, in the overall population, women with BRCAm and HRD, and treated with PARPi had the highest SUCRA value, suggesting it as a more favorable treatment option for preventing ovarian cancer progression in the first line setting [strength of evidence IIB].”
BackgroundInfluentialYin, S., Wang, P., Xu, X., Tan, Y., Huang, J., & Xu, H. (2019) · Journal of Cancer · doi:10.7150/jca.30456 · Cited by 11
“With the highest probability of being the best treatment, the level of evidence for neoadjuvant CRT was downgraded from high to moderate, as the CrI crossed the unit [21].”
BackgroundRamasamy, K., Dhanasiri, S., Thom, H. H. Z., Buchanan, V., Robinson, S. K., D’Souza, V., & Weisel, K. C. (2019) · Leukemia & lymphoma/Leukemia and lymphoma · doi:10.1080/10428194.2019.1683736 · Cited by 11
“of trials comparing the treatments [12,13].”
Reactive air surfaces for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., Leung, V., & McInnes, E. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013622.pub2 · Cited by 12
Liu, S., Chen, J., He, Y., Wu, L., Lai, J., Zuo, J., Yang, L., & Guo, X. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017099 · Cited by 11
“Quality of evidence We will also assess the quality of evidence for the main outcomes with the GRADE (the Grading of Recommendations Assessment, Development and Evaluation) approach.(45) The five items will be investigated, including limitations in study design, inconsistency, imprecision, indirectness and publication…”
MethodDressings and topical agents for treating pressure ulcers
Westby, M. J., Dumville, J. C., Soares, M. O., Stubbs, N. C., Norman, G., & Foley, C. N. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011947 · Cited by 13
Comparative efficacy of topical interventions for facial photoaging: a network meta-analysis
Lin, L., Chen, X., Liu, C., Wang, Q., Lian, W., Xu, X., Guo, Y., Zhong, H., Zhong, J., Zhao, N., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-12597-0 · Cited by 13
Zheng, X., Wang, R., Giri, M., Duan, J., Ma, M., & Guo, S. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.1026234 · Cited by 10
“The weight contribution matrix was constructed to assess the information contribution of direct evidence to entire NMA estimates (27).”
BackgroundTian, Y., Gao, W.-Y., Tian, X.-R., & Wang, Z. (2022) · Therapeutic Hypothermia and Temperature Management · doi:10.1089/ther.2022.0032 · Cited by 10
“We assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework (Salanti et al. , 2014).”
Xu, H., Du, Y., Wang, Q., Chen, L., Huang, J., Liu, Y., Zhou, C., & Biao, D. (2022) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2022.103394 · Cited by 13
Local Insulin Improves Wound Healing: A Systematic Review and Bayesian Network Meta-Analysis
Ramírez-GarcíaLuna, J. L., Rangel-Berridi, K., Bergeron, A., Kolosovas‐Machuca, E. S., Wang, S. C., Berry, G. K., & Martínez‐Jiménez, M. A. (2023) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000010432 · Cited by 17
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.