Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 701–800 of 999
Tang, R., Gong, S., Li, J., Hu, W., Liu, J., & Liao, C. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1337616 · Cited by 12
“Network meta-analysis (NMA) has been proposed to be the highest level of evidence in the treatment guideline (Salanti et al., 2014).”
Yang, Y., Gu, W., Xu, S., Wang, S., Shi, H., Zhang, L., Meng, X., Hong, F., & Du, Y. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-090497 · Cited by 14
Yu, Z., Yang, X., Ma, T., Qin, F., Ren, L., Gao, S., Chen, J., & Liu, X. (2024) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2024.12.016 · Cited by 10
Mak, I. E. K., Yao, Y., Ng, M. T. T., & Kim, J. E. (2025) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2025.2452362 · Cited by 14
“…for all outcomes was evaluated in accordance to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for Network Meta-Analysis, where the estimates were rated as high, moderate, low, or very low certainty (Brignardello-Petersen et al. 2018; Salanti et al. 2014).”
Chen, C.-S., Wen, L., Yang, F., Deng, Y.-C., Ji, J.-H., Chen, Z., Chen, Z., Chen, G., & Gu, J.-Y. (2025) · Journal of Chinese Integrative Medicine · doi:10.1016/j.joim.2025.04.008 · Cited by 11
Clephas, P. R. D., Orbach‐Zinger, S., Gosteli, M., Hoshen, M., Halpern, S. H., Hilber, N. D., Leo, C., & Heesen, M. A. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014818.pub2 · Cited by 24
Vacharanukrauh, P., Miller, K. J., Alif, S. M., Grace, F., & Rahman, M. A. (2025) · Breast Cancer Research and Treatment · doi:10.1007/s10549-025-07791-7 · Cited by 11
Answering complex hierarchy questions in network meta-analysis
Papakonstantinou, T., Salanti, G., Mavridis, D., Rücker, G., Schwarzer, G., & Nikolakopoulou, A. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01488-3 · Cited by 9
“Preliminary suggestions have associated the uncertainty of the entire treatment hierarchy with the shape of the rankograms: rankograms which show large differences between each treatment being at each rank indicate a precise treatment hierarchy, while flat rankograms reflect uncertainty in the treatment hierarchy [27].”
BackgroundSelf-citationXiao, L., Niu, L., Xu, X., Zhao, Y., Yue, L., Liu, X., & Li, G. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.830030 · Cited by 10
“After that, for each treatment comparison, the comparison-adjusted funnel plot were used to assess the presence of small-study effects and publication bias if more than 10 studies were included (Salanti et al., 2014).”
MethodVishwanath, V., Akseer, S., Frankfurter, C., Reza, S. J., Farrell, A., Horlick, E. M., & Abrahamyan, L. (2022) · Archives of cardiovascular diseases · doi:10.1016/j.acvd.2022.09.002 · Cited by 10
Rizzuto, A., Pirrera, A., Gigliotta, E., Mancuso, S., Vullo, C., Camarda, G. M., Rotolo, C., Roppolo, A., Spoto, C., Gentile, M., et al. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24129930 · Cited by 10
“Concurrently, we assessed the heterogeneity and consistency of our analysis by evaluating the log of the ratio of 2 odds ratios (RoR) from direct and indirect evidence in the loop (ifplot command in STATA) [30,31].”
MethodGupta, A. K., Wang, T., Bamimore, M. A., Piguet, V., & Tosti, A. (2023) · Journal of Cosmetic Dermatology · doi:10.1111/jocd.15903 · Cited by 10
Generalised decision curve analysis for explicit comparison of treatment effects
Hozo, I., & Djulbegović, B. (2023) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13915 · Cited by 12
Zhao, J. J., Fong, K. Y., Chan, Y. H., Tey, J. C. S., Dawood, S. S., Lee, S. C., Finn, R. S., Sundar, R., & Lim, J. S. J. (2023) · Cancers · doi:10.3390/cancers15184558 · Cited by 14
“The The transitivity assumption was evaluated clinically [16]. transitivity assumption was evaluated clinically [16].”
BackgroundAiolfi, A., Bona, D., Bonitta, G., Bonavina, L., & Gastric Ischemic Conditioning (GIC) International Collaborative Group (2023) · Annals of Surgery · doi:10.1097/sla.0000000000006124 · Cited by 10
“37 Statistical analyses were carried out using JAGS and R ‐ Cran 3.4.3 (Distributed Statistical Computing; Vienna, Austria).”
MethodInfluentialDagher, R., Gad, M., de Santana, P. D. S., Sadeghi, M. A., Yewedalsew, S. F., Gujar, S. K., Yedavalli, V. S., Köhler, C. A., Khan, M., Távora, D. G. F., et al. (2024) · Journal of Neuro-Oncology · doi:10.1007/s11060-023-04528-8 · Cited by 10
“The ranking was also assessed by visually inspecting the SUCRA curve for each imaging modality [28, 29].”
Chen, P., Ho, C., Lin, C. S., Wu, Y., Huang, T., Tu, Y., & Lee, C. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8050737 · Cited by 11
Zhu, Y., Zou, Z., Huang, Y., Zhang, L., Chen, H., Li, Y., Liu, C., Li, X., Xu, D., & Zeng, Q. (2019) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezz335 · Cited by 11
Kataoka, Y., Luo, Y., Chaimani, A., Ōnishi, A., Kimachi, M., Tsujimoto, Y., Murad, M. H., Li, T., Cipriani, A., & Furukawa, T. A. (2020) · Archives of Osteoporosis · doi:10.1007/s11657-020-0697-8 · Cited by 13
Self-citationNetwork meta-analysis of antiepileptic drugs in focal drug-resistant epilepsy
Hu, T. Y., Wang, H., Zhang, W. P., Tian, R. F., Lei, G., Deng, Y. C., & Xing, J.-L. (2020) · Epilepsy Research · doi:10.1016/j.eplepsyres.2020.106433 · Cited by 12
“ranks based on rankograms, the surface under the cumulative ranking curve (SUCRA) which take into account the estimated effect sizes and their accompanying uncertainty (Salanti et al., 2011; Salanti et al., 2014).”
The ANiMALIA investigators, Wewege, M. A., Bagg, M. K., Jones, M. D., & McAuley, J. H. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01506-3 · Cited by 10
“We will present a contributions matrix to indicate the weighting of direct evidence contributions to each NMA effect size, which will also be used to evaluate the confidence in the overall evidence (see the ‘Confidence in cumulative evidence’ section) [72].”
BackgroundMethodCheng, H., Elbers, R. G., Higgins, J. P. T., Taylor, A., MacArthur, G. J., McGuinness, L. A., Dawson, S., López-López, J. A., Cowlishaw, S., Hickman, M., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0462-2 · Cited by 11
“The NMA will be based on random-effects models to account for heterogeneity in treatment effects within each comparison, assuming a common heterogeneity variance (τ(2)) across all comparisons [35].”
MethodSelf-citationOstinelli, E. G., D’Agostino, A., Shokraneh, F., Salanti, G., & Furukawa, T. A. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032726 · Cited by 13
Self-citationAtypical antipsychotics for autism spectrum disorder: a network meta-analysis
Meza, N., Rees, R., Liquitay, C. M. E., Franco, J. V. A., Sguassero, Y., Williams, K. J., Pringsheim, T. M., Rojas, V., & Madrid, E. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014965 · Cited by 15
Walsh, G. S., Delextrat, A. A., & Bibbey, A. (2023) · Maturitas · doi:10.1016/j.maturitas.2023.107790 · Cited by 16
Wang, Y., Xia, R., Peričić, T. P., Bekkering, G. E., Hou, L., Bała, M. M., Gao, Y., Wu, M., Gloss, D. S., Siemieniuk, R. A. C., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-075212 · Cited by 12
Jiang, Z., Deng, B., Zhang, Y., Li, Y., Yu, Z., Deng, H., & Chen, Y. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-074188 · Cited by 9
“28 The certainty of the NMA estimates was rated as ‘high’, ‘moderate’, ‘low’ or ‘very low’ based on considerations of risk of bias, inconsistency, indirectness, imprecision and publication bias.”
BackgroundWang, J., Zhang, H., Wang, T., Liu, G., Teng, Y., Wang, J., Zhang, Q., Yan, S., & Ji, B. (2024) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2024.11.001 · Cited by 10
Gao, W., Zhang, Q., Zhang, L., Xu, M., Zhu, Y., & Han, J. (2025) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2025.102798 · Cited by 9
Surgical approaches to varicocele: a systematic review and network meta-analysis
Lu, L.-J., Xiong, K., Yuan, S., Che, B., Zhai, J., Wu, C., Zhang, Y., Zhang, H., & Tang, K. (2025) · Asian Journal of Andrology · doi:10.4103/aja202541 · Cited by 9
“Additionally, confidence in network meta-analysis (CINeMA) 14 was leveraged to assess the evidence quality of the entire NMA based on six aspects: within-study bias, between-study bias, indirectness, imprecision, heterogeneity, and inconsistency.”
Brupbacher, G., Gerger, H., Wechsler, M., Zander-Schellenberg, T., Straus, D., Porschke, H., Gerber, M., von Känel, R., & Schmidt‐Trucksäss, A. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1018-4 · Cited by 11
“The confidence in the network meta-analyses will be estimated using the Confidence in Network Meta-Analysis (CINeMA) framework [110].”
MethodZhang, K., Zhang, Y., Liu, X., Li, A., Gao, M., Hou, J., Guo, C., Zhang, T., Wu, H., Chen, G., et al. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.603068 · Cited by 9
Tawfik, G. M., Biala, M. I., Yousef, Y. M., Tiwari, R., Dobs, M., Lotfy, C. I., Farrag, D. A., Hue, A. T., Yotsu, R. R., & Huy, N. T. (2021) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2021.07.032 · Cited by 11
Fu, H., Zhou, H., Tang, Y., Li, J., Zhang, D., Ding, S.-Y., Huang, Q., & Wang, C. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-052137 · Cited by 9
Helali, A. E., Wong, C. H. L., Choi, C., Chan, W., Dickson, N., Siu, S. W. K., Chan, K. K. L., Ngan, H., Ngan, R. K. C., & Kennedy, R. D. (2022) · Scientific Reports · doi:10.1038/s41598-022-07731-1 · Cited by 12
Bouché, P.-A., Corsia, S., Hallé, A., Gaujac, N., & Nizard, R. (2023) · The Knee · doi:10.1016/j.knee.2023.01.003 · Cited by 10
Network Meta-Analysis of Behavioral Programs for Smoking Cessation in Healthy People
Xu, M., Guo, K., Shang, X., Zhou, L., Fenfen, E., Yang, C., Wu, Y., Li, X., & Yang, K. (2023) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2023.02.032 · Cited by 12
Jiang, X., Zhang, L., Liu, H., Su, H., Jiang, J., Chen, Q., Wang, Q., Qu, X., Sun, W., & Bi, H. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-068019 · Cited by 9
Tong, K., Chen, M., Wang, D., Dai, H., Peng, J., Zhang, J., Zhou, J., Chang, Y., & Huang, W. (2024) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-023-03616-y · Cited by 12
“1–4), and these contributions can be used to assess the quality of evidence for inferences related to the network as a whole [30, 31].”
Druyts, E., Eapen, S., Wu, P., & Thorlund, K. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-116 · Cited by 10
“We will include (1) the key role of indirect comparisons, (2) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size, (3) the importance of the transitivity assumption to the validity of network meta-analysis, and (4) the possibility of disagreement between direct evid…”
BackgroundInterventions for the treatment of heavy menstrual bleeding
Rodriguez, M. B., Dias, S., Brown, J., Wilkinson, J., Lethaby, A. E., Lensen, S. F., Jordan, V., WISE, M. R., & Farquhar, C. M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013180 · Cited by 9
Caro, P., Turner, W., & Macdonald, G. M. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013361 · Cited by 10
“DOI: 10.1016/j.jclinepi.2010.03.016; PUBMED: 20688472 Salanti 2014 Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JPT. Evaluating the quality of evidence from a network meta-analysis.”
BackgroundMethodInfluentialde Zoete, R. M. J., McAuley, J. H., Armfield, N. R., & Sterling, M. M. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034846 · Cited by 10
“32 Confidence in cumulative evidence Based on study limitations, imprecision, heterogeneity indirectness and publication bias, 33 the overall quality of evidence will be assessed with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework, 34 which has been adapted…”
Pharmacological treatment of neuropsychiatric symptoms of dementia: a network meta-analysis protocol
Huang, Y., Dou, K., Zhong, X.-L., Shen, X., Chen, S.-D., Li, H., Chen, K., Cui, M., Dong, Q., Tan, L., et al. (2020) · Annals of Translational Medicine · doi:10.21037/atm-20-611 · Cited by 7
Yang, S., He, W., Zhao, L., & Mi, Y. (2022) · PLoS ONE · doi:10.1371/journal.pone.0267025 · Cited by 10
“Furthermore, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework was used to assess the quality of evidence contributing to each network estimate, which characterizes the quality of a body of evidence for the outcomes based on study limitations, imprecision, inconsistency, indirectn…”
MethodHuang, P., Chen, Y., Zhang, H., Chen, B., Zhao, S., Feng, Y., Lei, S., & Wu, Q. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.850221 · Cited by 14
“, pairwise comparison and rank probability among the interventions in each outcome (Salanti et al., 2014).”
BackgroundUpdate on vitamin C administration in critical illness
Fujii, T., Lankadeva, Y. R., & Bellomo, R. (2022) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000000951 · Cited by 10
“When three or more interventions are investigated for one condition, network meta-analysis (NMA) compares them simultaneously in a single analysis [18].”
BackgroundFujii, T., Belletti, A., Carr, A. C., Furukawa, T. A., Luethi, N., Putzu, A., Sartini, C., Salanti, G., Tsujimoto, Y., Udy, A. A., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-033458 · Cited by 8
Self-citationClegg, A. J., Iheozor‐Ejiofor, Z., Gordon, M. A., MacDonald, J. K., & Akobeng, A. K. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013210 · Cited by 9
Radadiya, D. K., Brahmbhatt, B. S., Reddy, C. M., & Devani, K. (2021) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000001523 · Cited by 28
Zhou, T., Bodawatta, K. H., & Jiang, A. (2023) · Avian Research · doi:10.1016/j.avrs.2023.100086 · Cited by 10
Reporting quality of systematic reviews with network meta-analyses in Endodontics
Nagendrababu, V., Narasimhan, S., Faggion, C. M., Dharmarajan, L., Jacob, P. S., Gopinath, V. K., & Dummer, P. M. H. (2023) · Clinical Oral Investigations · doi:10.1007/s00784-023-04948-w · Cited by 10
Diagnostic accuracy of tests for tuberculous pericarditis: A network meta-analysis
Pervez, A., Hasan, S. U., Hamza, M., Asghar, S., Qaiser, M. H., Zaidi, S., & Mustansar, I. (2023) · Indian Journal of Tuberculosis · doi:10.1016/j.ijtb.2023.05.013 · Cited by 10
Carrier, F. M., Girard, M., Zuo, R. M., Ziegler, D., Trottier, H., & Chassé, M. (2023) · Transplantation · doi:10.1097/tp.0000000000004744 · Cited by 11
“Two studies comparing different vasodilator infusions reported the incidence of graft dysfunction 64,66 and 1 reported the Fenoldopam 45 [42–50] 11 (24) – 72 (18) 46 (100) 0 (0) – – 535 (155) 0 (0) Low-dose dopamine 50 [46–54]”
BackgroundMethodCheng, H.-Y., Liang, C.-W., Lee, Y.-H., Vitoonpong, T., Liao, C., & Huang, S. (2024) · EFORT Open Reviews · doi:10.1530/eor-23-0136 · Cited by 8
“(11).”
Pornsuriyasak, P., Sa-nguansai, S., Thadanipon, K., Numthavaj, P., McKay, G. J., Attia, J., & Thakkinstian, A. (2025) · BMC Medicine · doi:10.1186/s12916-025-03847-z · Cited by 10
Hudson, J., Imamura, M., Robertson, C., Whibley, D., Aucott, L. S., Gillies, K., Manson, P. D., Dulake, D., Abhishek, A., Tang, N. K. Y., et al. (2025) · Arthritis Care & Research · doi:10.1002/acr.25505 · Cited by 12
“Two reviewers working together evaluated the certainty of the evidence included in the NMA, using the Con fi dence in NMA (CINeMA) approach, 18 which is broadly based on the Grading of Recommendations Assessment, Development and Evaluation framework.”
Bermpohl, F. M. G., & Martin, A. (2025) · Psychotherapy and Psychosomatics · doi:10.1159/000544825 · Cited by 14
“It is further possible to rank the ef fi cacy of the included conditions in NMA based on P-scores [41].”
He, Q., Bennett, A. N., Zhang, C., Zhang, J. Y., Tong, S., & Chan, K. H. K. (2025) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2025.03.040 · Cited by 10
Quan, Y., Lo, C. Y., Wolff, L., Wang, J., Olsen, K. N., & Thompson, W. F. (2025) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2025.105848 · Cited by 8
Ren, M., Xian, G., Tan, X., Sun, S., & Zhang, M. (2025) · Frontiers in Physiology · doi:10.3389/fphys.2025.1587876 · Cited by 10
“For the CSA outcome, the HFHP intervention showed greater improvement than the CG (WMD = 0.50, 95% CI: −0.10–1.11, P > 0.05), and HFHP intervention also outperformed other BFR-LI training regimens, though differences were not statistically Frontiers in Physiology 07 frontiersin.org Loop consistency…”
InfluentialChen, S. X., Luo, L., & Yuan, Y. (2025) · Clinical Rehabilitation · doi:10.1177/02692155251355083 · Cited by 9
“Subsequently, the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system 18 was employed to classify the overall quality of the evidence.”
Ding, H., Jiang, L., Seo, H., & Chun, B. (2025) · BMC Medicine · doi:10.1186/s12916-025-04495-z · Cited by 9
“Statistical significance was defined a priori as a 95% CrI that did not include zero, and precision was further graded according to CrI width as High (<0.2), Moderate (0.2–0.5), or Low (>0.5) [38, 39].”
InfluentialShinozaki, S., Watanabe, J., Kanno, T., Nakazawa, K., & Yano, T. (2026) · Digestive Endoscopy · doi:10.1111/den.70138 · Cited by 13
“By comparing the distributions of the effect modifiers, country, age, and experience of endoscopists, we evaluated transitivity across the different comparisons [19].”
Hasan, S. U., Pervez, A., Bhatty, S. A., Shamim, S., Naeem, A., & Naseeb, M. W. (2022) · European Journal of Neuroscience · doi:10.1111/ejn.15732 · Cited by 8
“…node split model and were calculated using DIC, D.bar and pD (Barnard et al., 2000; Jansen & Cope, 2012); because of the more complex data structure, the model we used denotes the variance of random effect as heterogeneous (Salanti et al., 2014), and the null hypothesis was used for heterogeneity.”
MethodAssessment of several postoperative protocols after rotator cuff repair: A network meta-analysis
Bouché, P.-A., Gaujac, N., Descamps, J., & Conso, C. (2022) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2022.103418 · Cited by 8
Wang, J., Li, W., Li, M., Wu, H.-B., & Qiu, Z.-K. (2023) · Medicine · doi:10.1097/md.0000000000034670 · Cited by 12
“Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework.[28] Each network estimate of primary outcomes according to the criteria: RoB, indirectness, inconsistency, imprecision, and publication bias were assessed.”
MethodDetermining the Optimal Treatment for Idiopathic Clubfoot
Wang, Y.-Y., Su, Y.-C., Tu, Y., Fang, C., Hong, C., Huang, M.-T., Lin, Y., Hsieh, M., Kuan, F., Shih, C., et al. (2023) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.22.01210 · Cited by 8
“The quality of evidence contributing to each NMA estimate was assessed using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework 36,37 .”
MethodGuo, C.-H., Zhong, J., Jin, X., & Zhang, J.-E. (2023) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2023.104680 · Cited by 8
Bertolini, F., Robertson, L., Bisson, J. I., Meader, N., Churchill, R. C., Ostuzzi, G., Stein, D. J., Williams, T., & Barbui, C. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013613.pub2 · Cited by 13
Liu, W., Li, W., Wu, B., Chen, H., Zhao, X., & Wangwan, H. (2024) · Journal of Cancer Survivorship · doi:10.1007/s11764-024-01699-9 · Cited by 11
Gibbs, V. N., Champaneria, R., Novak, A., Doree, C. J., Palmer, A. J. R., & Estcourt, L. J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013499 · Cited by 8
Kishi, T., Nosaka, T., Sakuma, K., Okuya, M., & Iwata, N. (2020) · Neuropsychopharmacology Reports · doi:10.1002/npr2.12131 · Cited by 7
Systemic corticosteroids for the prevention of bronchopulmonary dysplasia, a network meta-analysis
Hay, S., Ovelman, C., Zupancic, J. A. F., Doyle, L. W., Onland, W., Konstantinidis, M., Shah, P. S., & Soll, R. F. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013730 · Cited by 8
Ni, M., Wang, H., Wang, M., Zhou, W., Wu, J., Sun, B., Zhang, Q., Fan, X., Zhang, D., Jing, Z., et al. (2020) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2020/1824536 · Cited by 7
“Network meta-analysis (NMA) was developed from traditional meta-analysis, extending from the standard twoarm meta-analysis to indirect comparisons of multiple related but different interventions at the same time [31].”
MethodBisphosphonates as Supplement to Dental Treatment: A Network Meta-Analysis
Zymperdikas, V. F., Yavropoulou, M. P., Kaklamanos, E. G., & Papadopoulos, M. A. A. P. (2020) · Journal of Dental Research · doi:10.1177/0022034520972945 · Cited by 8
“With regard to the outcomes assessed via network meta-analyses, the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach was used for the rating of the overall quality of clinical recommendations according to the methodology suggested by Salanti et al. (2014).”
MethodShen, Y., Jin, S., Lin, C., Shen, J., Hong, Z., Xue, S., & Guo, D. (2020) · Annals of Translational Medicine · doi:10.21037/atm-20-4451 · Cited by 7
“Quality of evidence We will assess the certainty of evidence contributing to each network estimate using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to reflect specific issues from the NMA (36).”
MethodAfach, S., Evrenoglou, T., Oubaya, N., Cleach, L. L., & Sbidian, É. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.01.013 · Cited by 13
Tehrani, H. D., Yamini, S., & Vazsonyi, A. T. (2023) · Journal of Experimental Criminology · doi:10.1007/s11292-023-09562-0 · Cited by 15
“The certainty (credibility or quality) of evidence in network estimates of the outcomes was evaluated based on the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) ratings (Salanti et al., 2014).”
Li, R., Xu, C., Zhong, P.-Y., Wang, K., Luo, Y., Xiao, L., Dai, X., Han, J., & Zhang, X. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1181160 · Cited by 10
“Besides, we applied Grades of Recommendations Assessment, Development and Evaluation (GRADE) to evaluate the quality of outcomes according to direct, indirect and network comparison, respectively (Puhan et al., 2014; Salanti et al., 2014).”
MethodPharmacological treatment of neuropsychiatric symptoms of dementia: a network meta-analysis.
Huang, Y.-Y., Teng, T., Giovane, C., Wang, R.-Z., Suckling, J., Shen, X.-N., Chen, S.-D., Huang, S., Kuo, K., Cai, W.-J., et al. (2023) · Age and Ageing · doi:10.1093/ageing/afad091 · Cited by 8
Yan, Y., Qiu, S., Deng, L., Tang, X., Li, X., Wei, Q., & Fu, P. (2018) · BMC Nephrology · doi:10.1186/s12882-018-1076-1 · Cited by 9
Evidence synthesis in pulmonary arterial hypertension: a systematic review and critical appraisal
Schlueter, M., Beaudet, A., Davies, E. W., Gurung, B., & Karabis, A. (2020) · BMC Pulmonary Medicine · doi:10.1186/s12890-020-01241-4 · Cited by 9
Foam surfaces for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013621 · Cited by 8
Mavridis, D., Palmer, S. C., & Strippoli, G. F. (2016) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2016.02.031 · Cited by 8
Musa, R., Elamin, D., Barrie, R., & Kimmie‐Dhansay, F. (2024) · JMIR Research Protocols · doi:10.2196/50479 · Cited by 11
“The results of the meta-analysis will be analyzed using the Confidence In Network Meta-Analysis (CINeMA) tool if there are more than 2 interventions included in the analysis; CINeMA is a software based on the framework developed by Salanti [34] and refined by Nikolakopoulou [35-37].”
MethodShortest path or random walks? A framework for path weights in network meta‐analysis
Rücker, G., Papakonstantinou, T., Nikolakopoulou, A., Schwarzer, G., Galla, T., & Davies, A. L. (2024) · Statistics in Medicine · doi:10.1002/sim.10177 · Cited by 7
“The first approach to this aim in the framework of a frequentist NMA approach was made in a paper by Salanti et al. 1 It was based on standardizing the rows of the hat matrix, a structure which maps the observed treatment effect estimates to their NMA counterparts.”
Tian, S., Liang, Z., Qiu, F., Yu, Y., Mou, H., Zhang, G., & Zhang, H. (2024) · Applied Psychology Health and Well-Being · doi:10.1111/aphw.12573 · Cited by 11
Cheng, X., Sun, S., Chen, M., Zhou, X., Rao, M., Guo, D., Xie, J., Huang, Q., & Su, L. (2024) · Obesity Reviews · doi:10.1111/obr.13834 · Cited by 8
Jiao, X., Du, M., Li, Q., Huang, C., Ding, R., & Wang, W. (2024) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-024-05569-y · Cited by 7
“The quality of evidence for primary outcomes was assessed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) framework [ 10].”
Bittencourt, J. V., Corrêa, L. A., Pagnez, M. A. M., do Rio, J. P. M., Telles, G. F., Mathieson, S., & Nogueira, L. A. C. (2024) · PLoS ONE · doi:10.1371/journal.pone.0313025 · Cited by 11
Wu, W., Wu, W.-H., Lee, M., Wu, P.-Y., Tu, Y., Lin, H. H., Chan, C., & Huang, C. (2025) · Sports Medicine · doi:10.1007/s40279-025-02179-5 · Cited by 8
“These contributions are influenced by the variances of direct estimates and their proximity to the target comparison [64].”
Shi, A., Shu, Y., Haddad, J. E., Wu, S., Smayra, K., Sudesh, S. M., Mourad, M. M., Farzad, A., Yap, N., Andrikopoulou, E., et al. (2025) · Dermatology and Therapy · doi:10.1007/s13555-025-01472-5 · Cited by 7
“The certainty of evidence for each outcome (body of evidence) was assessed according to CiNeMa [30] and includes six domains: within-study bias, across-studies bias, indirectness, imprecision, heterogeneity, and incoherence.”
Santana, B. R., García, J. S., y, A. J.-R., Cantolla, P., Fernández, M. G., López-Sendón, J., Tardif, J., Moreno, R., & Fernández‐Velasco, M. (2025) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2025.120443 · Cited by 7
Ambrosio, L., Schol, J., Tamagawa, S., Muthu, S., Sakai, D., Papalia, R., Vadalà, G., & Denaro, V. (2025) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.24.01205 · Cited by 11
Zhang, L., Wang, F., Tashiro, S., & Liu, P. J. (2025) · Clinical Nutrition · doi:10.1016/j.clnu.2025.08.005 · Cited by 7
Tian, S., Che, W., Liang, Z., Qiu, F., Yu, Y., & Wang, X. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.120604 · Cited by 11
Yin, S., Han, Y., Liu, Y., Chen, B., Fu, Z., Sheng, S., Wang, J., Shen, C., Wang, X., & Jia, Y. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.972498 · Cited by 7
“Frontiers in Oncology 14 understand that high-quality evidence might be difficult to obtain because grade judgments can be overly cautious (49).”
BackgroundZitikyte, G., Roy, D. C., Tran, A., Fernando, S. M., Rosenberg, E., Kanji, S., Engels, P. T., Wells, G. A., & Vaillancourt, C. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000875 · Cited by 9
“Parameter Soltani et al (12) SanchezIzquierdoRiera et al (43) Kelly et al (44) Camps et al (30) Khallaf et al (45) Winings et al (13)”
BackgroundMethod
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.