Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 501–600 of 999
Qu, Q., Sun, J., Zhang, Z., Kan, J., Wu, L., Li, F., & Wang, R. (2021) · Journal of Cardiovascular Electrophysiology · doi:10.1111/jce.15269 · Cited by 21
“The certainty of the evidence for outcome measures was assessed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for network meta‐ analyses, which is associated with estimates from direct and indirect comparisons.(14) Our certainty assessment reflects whether the results are cl…”
MethodLu, X., Cai, H., Li, Y. R., Zheng, X., Yun, J., Li, W., Geng, X., Kwon, J., & Jiang, H. B. (2022) · Bioinorganic Chemistry and Applications · doi:10.1155/2022/4529520 · Cited by 22
The complexity underlying treatment rankings: how to use them and what to look at
Chiocchia, V., White, I. R., & Salanti, G. (2022) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2021-111904 · Cited by 21
Self-citationMiller, K. J., Areerob, P., Hennessy, D., Gonçalves-Bradley, D. C., Mesagno, C., & Grace, F. (2021) · F1000Research · doi:10.12688/f1000research.27123.2 · Cited by 20
Chao, Y.-C., Chen, P., Su, W., Yeh, H.-W., Su, C.-C., Wu, Y., Chiang, H.-S., Jhou, H., & Shieh, Y. (2022) · Journal of Dental Sciences · doi:10.1016/j.jds.2022.05.013 · Cited by 27
Zhang, X., Xiao, L., Niu, L., Tian, Y., & Chen, K. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.964616 · Cited by 20
“After that, publication bias were reflected by funnel plots (31, 32).”
MethodGalli, M., Benenati, S., Zito, A., Capodanno, D., Zoccai, G., Ortega-Paz, L., Iaconelli, A., D'Amario, D., Porto, I., Burzotta, F., et al. (2022) · International Journal of Cardiology · doi:10.1016/j.ijcard.2022.10.023 · Cited by 16
Asimakidou, E., & Sidiropoulos, C. (2023) · Toxins · doi:10.3390/toxins15040256 · Cited by 23
“Additionally, the certainty of evidence from the network meta-analysis was rated with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [38].”
BackgroundMethodLight exposure therapy for myopia control: a systematic review and Bayesian network meta-analysis
Zaabaar, E., Zhang, X. J., Zhang, Y., Bui, C. H. T., Tang, F. Y., Kam, K. W., Szeto, S., Young, A. L., Wong, I. C. K., Ip, P., et al. (2023) · British Journal of Ophthalmology · doi:10.1136/bjo-2023-323798 · Cited by 20
Yi, Y., Zhao, W., Lv, S., Zhang, G., Rong, Y., Wang, X., Yang, J., & Li, M. (2024) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2024.07.011 · Cited by 25
Shamim, M. A., Patil, A., Amin, U., Roy, T., Tiwari, K., Husain, N., Kumar, J., Chenchula, S., Rao, P. P., Ganesh, V., et al. (2024) · Diabetes Obesity and Metabolism · doi:10.1111/dom.15777 · Cited by 26
“We used a comparison-adjusted funnel plot 45 to assess publication bias in the NMA.”
Treatment strategies for Graves’ ophthalmopathy: a network meta-analysis
Zhou, X., Zhou, D., Wang, J., & Chen, G. (2019) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2018-313697 · Cited by 21
Liu, T., Jin, B., Chen, J., Wang, H., Lin, S., Dang, J., & Li, G. (2020) · Therapeutic Advances in Medical Oncology · doi:10.1177/1758835920940927 · Cited by 23
Network meta-analysis reaches nutrition research
Schwingshackl, L., Buyken, A. E., & Chaimani, A. (2018) · European Journal of Nutrition · doi:10.1007/s00394-018-1849-0 · Cited by 21
Self-citationNonpharmacological Interventions for Pediatric Migraine: A Network Meta-analysis
Koechlin, H., Kossowsky, J., Lam, T. L., Barthel, J., Gaab, J., Berde, C. B., Schwarzer, G., Linde, K., Meißner, K., & Locher, C. (2021) · PEDIATRICS · doi:10.1542/peds.2019-4107 · Cited by 22
Wang, Z., He, Y. L., Zhang, X., & Luo, Z. (2021) · Frontiers in Pediatrics · doi:10.3389/fped.2021.749975 · Cited by 19
“The certainty of the evidence for the each outcome was evaluated using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework (28) with five criteria: study limitations, imprecision, indirectness, heterogeneity and inconsistency, and publication bias.”
MethodShi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013761 · Cited by 23
Huang, J., Fan, Y., Zhao, K., Yang, C., Zhao, Z., Chen, Y., Yang, J., Wang, T., & Qu, Y. (2022) · Neuromodulation Technology at the Neural Interface · doi:10.1016/j.neurom.2022.06.004 · Cited by 22
Theodoridis, X., Kalopitas, G., Vadarlis, A., Bakaloudi, D. R., Gkiourtzis, Ν., Dionysopoulos, G., Karanika, E., Tsekitsidi, E., & Chourdakis, M. (2022) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2022.108294 · Cited by 25
Kihal, A., Rodríguez-Prado, M., & Calsamiglia, S. (2023) · Journal of Dairy Science · doi:10.3168/jds.2022-23028 · Cited by 19
“ment between estimates within the same comparison (Salanti et al., 2014).”
BackgroundTan, L., Yan, W., Zhang, B., Zhao, Z., Lipowski, M., & Ossowski, Z. M. (2024) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2024.03.007 · Cited by 23
Risk-reducing medication for primary breast cancer: a network meta-analysis
Mocellin, S., Goodwin, A., & Pasquali, S. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012191 · Cited by 20
Tsoi, B., Akioyamen, L. E., Bonner, A., Frankfurter, C., Levine, M., Pullenayegum, E., Goeree, R. A., & O’Reilly, D. J. (2017) · Heart Lung and Circulation · doi:10.1016/j.hlc.2017.06.721 · Cited by 19
Liang, J., Li, J., Jia, R., Ying-quan, W., Wu, R.-K., Zhang, H., Hang, L., Xu, Y., & Pan, C. (2018) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2018.09.017 · Cited by 22
Rausa, E., Manfredi, R., Kelly, M. E., Bianco, F., Aiolfi, A., Bonitta, G., Zappa, M. A., & Lucianetti, A. (2020) · Journal of Laparoendoscopic & Advanced Surgical Techniques · doi:10.1089/lap.2020.0752 · Cited by 19
“20 All statistics analysis were computed using Jags and R-Cran softwars.”
Zeng, B., Zeng, B., Chen, Y., Hung, C., Sun, C., Cheng, Y., Stubbs, B., Carvalho, A. F., Brunoni, A. R., Su, K., et al. (2021) · International Journal of Obesity · doi:10.1038/s41366-021-00833-2 · Cited by 21
O'Donnell, J. P., Murphy, D., Ryan, É. J., Gasior, S. A., Sugrue, R., O’Neill, B. L., Boland, M. R., Lowery, A., Kerin, M. J., & McInerney, N. M. (2021) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2021.07.001 · Cited by 18
Schneider‐Thoma, J., Kapfhammer, A., Wang, D., Bighelli, I., Siafis, S., Wu, H., Hansen, W., Davis, J. M., Salanti, G., & Leucht, S. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01760-z · Cited by 20
Self-citationXu, G., Lin, M., Dai, X., & Hu, J. (2021) · Endocrinology Diabetes & Metabolism · doi:10.1002/edm2.314 · Cited by 17
Huang, X., Yuan, F., Chen, Y., Huang, Y., Yang, Z., Lin, L., & Yu, J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-051608 · Cited by 20
Gibbs, V. N., Geneen, L. J., Champaneria, R., Raval, P., Doree, C. J., Brunskill, S. J., Novak, A., Palmer, A. J. R., & Estcourt, L. J. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013499.pub2 · Cited by 22
Ding, X., Ma, Y., Ma, Y., Chen, C., Zhang, X., Duan, Q., & Ma, Y. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000630 · Cited by 26
Systemic corticosteroids for the prevention of bronchopulmonary dysplasia, a network meta-analysis
Hay, S., Ovelman, C., Zupancic, J. A. F., Doyle, L. W., Onland, W., Konstantinidis, M., Shah, P. S., & Soll, R. F. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013730.pub2 · Cited by 24
Zhou, S., Wang, M., Li, W., Zhang, Y., Zhao, T., Song, Q., & Cong, J. (2023) · Clinical Nutrition · doi:10.1016/j.clnu.2023.11.010 · Cited by 29
Lunny, C., Veroniki, A. A., Higgins, J. P. T., Dias, S., Hutton, B., Wright, J. M., White, I. R., Whiting, P. F., & Tricco, A. C. (2024) · Systematic Reviews · doi:10.1186/s13643-023-02388-x · Cited by 21
“Three CINeMA studies were similar but reported slightly different results: Nikolakopou-lou [23], Papakonstantinou [39] and Salanti [40].”
ResultSelf-citationFei, Q., Pan, J., Zhang, F., Lin, Y., & Yuan, T. (2024) · Critical Care Medicine · doi:10.1097/ccm.0000000000006227 · Cited by 18
“The Grade of Recommendations Assessment, Development, and Evaluation (GRADE) methodology, which is based on the aspects of study limitations, inconsistency, indi-rectness, imprecision, and publication bias, was used for grading the certainty of direct and network evidence (11).”
BackgroundMethodGibbs, V. N., Champaneria, R., Palmer, A. J. R., Doree, C. J., & Estcourt, L. J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013295 · Cited by 19
“Published by John Wiley & Sons, Ltd. by combining data from direct comparisons in clinical trials with missing comparison information in the network structure, and enables more precise estimates to be obtained by using data from across the network (Krahn 2013; Salanti 2014).”
Bonovas, S., Nikolopoulos, G. K., Piovani, D., González‐Lorenzo, M., Pantavou, K. G., Lytras, T., Peyrin‐Biroulet, L., & Danese, S. (2019) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14051 · Cited by 22
“The contribution of direct evidence to the mixed estimates was also estimated and plotted.(19,25) Probabilities of each drug being at a specific order, mean ranks of drugs, and surface under the cumulative ranking area (SUCRA) values,(18,19,21,26) were estimated with bootstrap resampling (10,000 times).”
BackgroundMethodThompson, T., Dias, S., Poulter, D. R., Weldon, S. M., Marsh, L. L., Rossato, C. J. L., Shin, J. I., Firth, J., Veronese, N., Dragioti, E., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01398-3 · Cited by 24
Smith, E., Cooper, N. J., Sutton, A. J., Abrams, K. R., & Hubbard, S. (2021) · BMC Public Health · doi:10.1186/s12889-021-10162-8 · Cited by 23
“Methods that aim to assess the quality of evidence supporting a NMA and how to quantify how much the evidence could change due to potential biases or sampling variation before the recommendation changes have been developed [28, 29].”
BackgroundKelley, G. A., & Kelley, K. S. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011258 · Cited by 21
“Confidence in cumulative evidence Strength of the body of evidence will be assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) instrument for network meta-analysis.(55) This includes evaluating the confidence in a specific pairwise effect estimate in a network meta-analysis (st…”
MethodBerglund, Z., Şimşek, Ş., & Feng, Y. (2024) · Foods · doi:10.3390/foods13050794 · Cited by 31
López‐Moreno, M., López‐Gil, J. F., Bravo‐Sànchez, A., Bertotti, G., & Roldán‐Ruiz, A. (2025) · Clinical Nutrition · doi:10.1016/j.clnu.2025.09.001 · Cited by 18
Mendes, A. J., Ribaldi, F., Sayin, O., Khachvani, G., Mulargia, R., Volpara, G., Remoli, G., Nencha, U., Giannoni‐Luza, S., Cappa, S. F., et al. (2025) · The Lancet Healthy Longevity · doi:10.1016/j.lanhl.2025.100762 · Cited by 27
Chen, J., Lv, M., Wu, S., Jiang, S., Xu, W., Qian, J., Chen, M., Fang, Z., Zeng, Z., & Zhang, J. (2021) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2021.10.054 · Cited by 21
Sun, Y., Ji, M., Leng, M., & Wang, Z. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.04.055 · Cited by 17
Ou-Yang, L.-J., Chen, P.-H., Lee, C.-H., Li, T.-Y., Wu, Y.-T., Jhou, H.-J., & Yu, T.-Y. (2022) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000002019 · Cited by 18
Network meta-analyses: Methodological prerequisites and clinical usefulness
Christofilos, S. I., Tsikopoulos, K., Tsikopoulos, A., Kitridis, D., Sidiropoulos, K., Stoikos, P., & Kavarthapu, V. (2022) · World Journal of Methodology · doi:10.5662/wjm.v12.i3.92 · Cited by 18
Sehmbi, H., Retter, S., Shah, U. J., Nguyen, D. L., Martin, J. E., & Uppal, V. (2022) · British Journal of Anaesthesia · doi:10.1016/j.bja.2022.08.042 · Cited by 18
Cavicchioli, M., Ogliari, A. C., Maffei, C., Mucci, C., Northoff, G., & Scalabrini, A. (2023) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.13547 · Cited by 21
Aiolfi, A., Sozzi, A., Bonitta, G., Lombardo, F., Cavalli, M., Campanelli, G., Bonavina, L., & Bona, D. (2023) · Surgical Endoscopy · doi:10.1007/s00464-023-10231-6 · Cited by 17
“The confidence in estimates of the outcome was assessed using Confidence in Network Meta‐Analysis (CINeMA) [27].”
MethodVerreault, M. D., Granger, É., Neveu, X., Delage, J. P., Bastien, C., & Vallières, A. (2023) · Journal of Sleep Research · doi:10.1111/jsr.14008 · Cited by 23
“This hypothesis is evaluated by checking the presence of heterogeneity and inconsistency in the network (Bucher et al., 1997; Lumley, 2002; Salanti et al., 2014).”
BackgroundMethodManara, M., Aiolfi, A., Sozzi, A., Calì, M., Grasso, F., Rausa, E., Bonitta, G., Bonavina, L., & Bona, D. (2024) · Cancers · doi:10.3390/cancers16091620 · Cited by 17
“Confidence in estimates of the outcome was evaluated using confidence in network meta-analysis (CINeMA) [48].”
MethodZhou, Z., Wan, Y., Li, C., Yuan, J., Gao, G., Cui, H., Li, J., & Zang, L. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.08.064 · Cited by 21
Haile, S. R., Guerra, B., Soriano, J. B., & Puhan, M. A. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0433-2 · Cited by 19
“Therefore one statistical approach to check for transitivity in our case is to explore the distribution variables giving case-mix across groups [56, 57], which we have adopted here using meta-regression.”
MethodBiologic-Induced Infections in Inflammatory Bowel Disease: The TNF-α Antagonists
McConachie, S. M., Wilhelm, S. M., Bhargava, A., & Kale-Pradhan, P. B. (2018) · Annals of Pharmacotherapy · doi:10.1177/1060028018754896 · Cited by 18
Luo, C., Wen, J., Sun, W., Li, T., Yu, X., Zhang, T., Zhou, X., Wu, W., & Li, R. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-023096 · Cited by 18
Pang, L., Liu, J., & Lv, X. (2019) · Medicine · doi:10.1097/md.0000000000016325 · Cited by 18
“The quality of evidences for each primary outcome will be assessed separately by 2 reviewers (PLJ and LXD) using the GRADE Working Group approach.([39,40]) The confidence in estimates will be categorized into 4 levels: high quality, moderate quality, low quality, and very low quality.”
Zhang, L., Li, X., Rüwald, J. M., Welle, K., Schildberg, F. A., & Kabir, K. (2020) · Technology and Health Care · doi:10.3233/thc-192078 · Cited by 17
“After all outcomes were evaluated and a table summarizing the findings was created using the GRADE system [17], following the four-step approach to rate the quality of evidence in each of the direct and mixed estimates based on methods developed by the GRADE working group [18,19].”
MethodGhasemi, K., Vaseghi, G., & Mansourian, M. (2020) · Journal of Oncology Pharmacy Practice · doi:10.1177/1078155220965674 · Cited by 17
“If there was heterogeneity (I(2)> 50%) in all of the comparative groups, the random-effect model was utilized to conduct network meta-analysis.(15,16) To assess any consistency between the direct and indirect comparisons, we applied the node-splitting model.”
MethodCao, J., Zhang, X., & Wang, Z. (2021) · Journal of Neurology · doi:10.1007/s00415-021-10487-9 · Cited by 23
Wang, X., He, X., Qi, F., Liu, J., & Wu, J. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.876386 · Cited by 17
“Additionally, we assessed the certainty of evidence contributing to focus comparison estimates (between IVTAFL, IVR, and IVC) of the primary outcomes (all patients at 1-year follow-up) with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework (Salanti et al., 2014).”
MethodTreatment of metastatic renal cell carcinoma in older patients: A network meta-analysis
Hale, P., Hahn, A. W., Rathi, N., Pal, S. K., Haaland, B. A., & Agarwal, N. (2018) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2018.05.010 · Cited by 17
Desborough, M., Estcourt, L. J., Chaimani, A., Doree, C. J., Hopewell, S., Trivella, M., Hadjinicolaou, A. V., Vyas, P., & Stanworth, S. J. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012055 · Cited by 19
Self-citationCheng, Y.-S., Tseng, P.-T., Wu, Y.-C., Tu, Y.-K., Wu, C.-K., Hsu, C.-W., Lei, W.-T., Li, D.-J., Chen, T.-Y., Stubbs, B., et al. (2021) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.190122 · Cited by 16
“24 We performed the full analytic procedure using STATA version 14.0 (www.stata.com/stata14/).”
MethodInfluentialCharitakis, E., Karlsson, L., Rizas, K. D., Almroth, H., Jönsson, A. H., Schweiler, J., Sideris, S. K., Tsartsalis, D., Dragioti, E., & Chaimani, A. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-041819 · Cited by 16
“The findings from this systematic review will be published in a peer-reviewed scientific journal, and data set will be made freely available on request.”
BackgroundSelf-citationZhang, L., Wang, F., Wang, R., Sun, B., & Liu, P. J. (2024) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2024.2409956 · Cited by 20
“The assessment of the certainty of evidence (CoE) was conducted using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) framework (Salanti et al. 2014).”
Chen, X., Dong, Y., Ye, M., Wang, X., Yan, J., Yao, Y., Zhou, Q., Qian, C., & Liu, Z. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-06735-1 · Cited by 22
“We assessed the confidence of evidence contributing to each network estimate using Confidence In Network Meta-Analysis (CINeMA) software [76].”
Botta, C., Gigliotta, E., Paiva, B., Anselmo, R., Santoro, M., Rodríguez‐Otero, P., Carlisi, M., Conticello, C., Romano, A., Solimando, A. G., et al. (2022) · Hematological Oncology · doi:10.1002/hon.3041 · Cited by 17
“2 odds ratios (RoR) from direct and indirect evidence in the loop (ifplot command in STATA).(10,11) RoR values close to 0 indicate that both direct and indirect evidence are in agreement.”
BackgroundMethodAnxiety and depression in lung cancer: effect of psychological interventions – network meta-analysis
Yuan, D., Huang, Y., Wu, J., Guo, Z., Li, S., & Zhang, Y. (2022) · BMJ Supportive & Palliative Care · doi:10.1136/spcare-2022-003808 · Cited by 20
Chen, X., Zhang, Q., Wen, S., Chen, F., & Zhou, C. (2022) · European Journal of Neurology · doi:10.1111/ene.15635 · Cited by 15
Caro, P., Turner, W., Caldwell, D. M., & Macdonald, G. M. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013361.pub2 · Cited by 37
Self-citationHowle, R., Ragbourne, S. C., Žolger, D., Owolabi, A. A., Onwochei, D. N., & Desai, N. (2024) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2023.111364 · Cited by 16
Bozkurt, E., Sijberden, J. P., Kasai, M., & Hilal, M. A. (2024) · HPB · doi:10.1016/j.hpb.2024.01.002 · Cited by 19
Silveira, A., Lima, C., Beaupré, L. A., Chepeha, J. C., & Jones, C. A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0294014 · Cited by 20
“The strength of evidence was measured by a synthesis of each outcome using the framework described by Salanti and colleagues [31] and implemented using the CINeMA TM [22,23] which allowed confidence in the results to be graded as high, moderate, low, and very low.”
Yu, H., Li, M., Qian, G., Yue, S., Ossowski, Z. M., & Szumilewicz, A. (2024) · Advances in Nutrition · doi:10.1016/j.advnut.2024.100253 · Cited by 24
“This approach is grounded in the Grading of Recommendations Assessment, Development, and Evaluation framework, as expounded by other investigators [29, 30].”
MethodMoriya, R. M., de Oliveira, C. E. C., Reiche, E. M. V., PASSINI, J. L. L., & Nunes, S. O. V. (2024) · Obesity Reviews · doi:10.1111/obr.13809 · Cited by 22
“Data was submitted to CINeMA 17,18 to investigate the relationship between ACE and adolescent with obesity and to assess the confidence in network analysis results.”
Qin, Z., Xie, S., Mao, Z., Liu, Y., Wu, J., Furukawa, T. A., Kwong, J. S. W., Tian, J., & Liu, Z. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-017392 · Cited by 19
“Two authors (JSWK and JT) will assess the quality of evidence using the GRADE framework, covering study limitations, inconsistency, indirectness, imprecision and publication bias.(40) The methods for rating the quality of direct comparisons are the same as the methods used in traditional meta-analyses, and the followi…”
MethodLaw, W. Z. W., Sara, R., Cameron, A. J. D., & Lightfoot, N. J. (2019) · Anaesthesia · doi:10.1111/anae.14864 · Cited by 18
Crescenzo, F. D., Amato, L., Cruciani, F., Moynihan, L. P., D’Alò, G. L., Vecchi, S., Saulle, R., Mitrova, Z., Franco, V. D., Addis, A., et al. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.649472 · Cited by 16
“Various other pharmacological agents have been touted as potential treatments for Covid-19, with a preponderance for established antiviral drugs licensed in the treatment of other infections (Sanders et al., 2020).”
BackgroundDai, R., Cao, Y., Zhang, H., Zhao, N., Ren, D., Jiang, X., Zheng, G., Bao, S., Yan, X., & Fan, J. (2021) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2021/9997516 · Cited by 17
“Finally, two independent reviewers evaluated the quality of direct and indirect evidence using the recommended rating, development, and rating methods [20].”
BackgroundYi, L.-J., Tian, X., Chen, M., Lei, J.-M., Xiao, N., & Herrera, M. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.731196 · Cited by 23
“GRADE Evaluation on Quality of Evidence We evaluated the certainty of evidence contributing to all network estimates of the primary outcomes by using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework (66).”
MethodSullivan, S. M., Coyle, D., & Wells, G. (2014) · PLoS ONE · doi:10.1371/journal.pone.0113277 · Cited by 16
Liu, S., Wang, Y., Zhu, X., Chen, F., & Shi, Y. (2024) · BMC Pulmonary Medicine · doi:10.1186/s12890-024-03429-4 · Cited by 15
Samara, M. T., Lappas, A. S., Pinioti, E., Glarou, E., Fober, I., Christogiannis, C., Siafis, S., Christodoulou, G. N., Helfer, B., Mavridis, D., et al. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103291 · Cited by 18
“30,31”
Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis
Sbidian, É., Chaimani, A., Guelimi, R., Tai, C., Beytout, Q., Choudhary, C., Nkusu, A. M., Ollivier, C., Samaran, Q., Hughes, C., et al. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011535.pub7 · Cited by 42
“The latter allows assessing transitivity, i.e. whether there were important differences between the trials evaluating different comparisons other than the treatments being compared (Salanti 2014).”
InfluentialSelf-citationAntidepressants and Risk of Sudden Cardiac Death: A Network Meta-Analysis and Systematic Review
Prasitlumkum, N., Cheungpasitporn, W., Tokavanich, N., Ding, K. R., Kewcharoen, J., Thongprayoon, C., Kaewput, W., Bathini, T. J., Vallabhajosyula, S., & Chokesuwattanaskul, R. (2021) · Medical Sciences · doi:10.3390/medsci9020026 · Cited by 24
“Moreover, the Grading of Recommendations Assessment and the Development and Evaluation (GRADE) framework was performed to assess the certainty of information accounted for the network estimates of the main outcomes from individual studies [26].”
MethodZhang, X., Huai, Y., Wei, Z., Yang, W., Xie, Q., & Li, Y. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.1007702 · Cited by 16
“With Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework, we additionally evaluated the certainty of evidence contributing to network estimates for the main outcomes (20).”
MethodSun, K., Wu, Y., Wu, L., & Shen, B. (2023) · Chinese Medical Journal · doi:10.1097/cm9.0000000000002183 · Cited by 16
Marzo, V. D., Savarese, G., Porto, I., Metra, M., & Ameri, P. (2023) · Journal of Cardiovascular Medicine · doi:10.2459/jcm.0000000000001504 · Cited by 16
Corticosteroid Injection Methods for Frozen Shoulder: A Network Meta-analysis
Liang, C.-W., Cheng, H.-Y., Lee, Y.-H., Liao, C., & Huang, S. (2024) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2024.01.003 · Cited by 16
Yao, Y., Huang, V., Seah, V., & Kim, J. E. (2024) · Nutrition Reviews · doi:10.1093/nutrit/nuae086 · Cited by 17
“To make inferences about the quality of evidence from the NMA, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach, which was suggested by Salanti et al, 27 was applied for the previous selected outcomes.”
Valisena, S., Azogui, B., Nizard, R., Tscholl, P. M., Cavaignac, É., Bouché, P.-A., & Hannouche, D. (2024) · EFORT Open Reviews · doi:10.1530/eor-23-0089 · Cited by 15
“The CINeMA app (https://cinema.ispm.unibe.ch/), which applies the modification of the GRADE framework for network meta-analysis (25), was used for the evaluation of confidence in the NMA findings.”
MethodVita, G., Pollini, D., Canozzi, A., Papola, D., Gastaldon, C., Correll, C. U., Barbui, C., & Ostuzzi, G. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.116124 · Cited by 17
“Confidence in evidence was evaluated using the Confidence in Network Meta-Analysis (CINeMA) methodology (Salanti et al., 2014).”
MethodSedation for awake tracheal intubation: A systematic review and network meta‐analysis
El‐Boghdadly, K., Desai, N., Jones, J. B., El-Ghazali, S., Ahmad, I., & Sneyd, J. R. (2024) · Anaesthesia · doi:10.1111/anae.16452 · Cited by 20
Armoiry, X., Tsertsvadze, A., Connock, M. J., Royle, P., Meléndez‐Torres, G. J., Souquet, P.-J., & Clarke, A. (2018) · PLoS ONE · doi:10.1371/journal.pone.0199575 · Cited by 16
Dang, F., Li, H., Wang, R., Wu, Q., Chen, H., Ren, J.-J., & Tian, J. (2019) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2019.08.017 · Cited by 18
Costa, G. D. M., Ziegelmann, P. K., Zanatta, F. B., Martins, C. C., Costa, P. D. M., & Mello, C. F. (2022) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2022.110557 · Cited by 15
Rausa, E., Ferrari, D., Kelly, M. E., Aiolfi, A., Vitellaro, M., Rottoli, M., Bonitta, G., & Bona, D. (2023) · Langenbeck s Archives of Surgery · doi:10.1007/s00423-023-02774-y · Cited by 16
“The confidence in estimates of the outcome was assessed using Confidence in Network Meta-Analysis (CINeMA) [29].”
N. Fountoulakis, K., S. McIntyre, R., & F. Carvalho, A. (2015) · Current Neuropharmacology · doi:10.2174/1570159x13666150630174343 · Cited by 15
“Advantages • Compared to conventional pairwise meta-analysis, NMA allows the incorporation of both direct as well as indirect sources of evidence; • Network meta-analysis allows a probability-based rank order of different treatments in terms of safety and efficacy; • Network meta-analysis may inform future research di…”
BackgroundInfluential
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.