Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 901–999 of 999
Li, P., Wang, W., Tao, Y., Tan, X., Li, Y., Mao, Y., Gao, L., Feng, L., Zhan, S., & Sun, F. (2023) · Chinese Medical Journal · doi:10.1097/cm9.0000000000002567 · Cited by 5
“The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) process was used to rate the quality of evidence for any significant result regarding primary outcomes in the NMA.”
MethodHasan, S. U., Pervez, A., Shah, A. A., Shah, S. D., Aslam, M., Arshad, A., Rajput, A. S., & Zubair, M. M. (2023) · Perfusion · doi:10.1177/02676591231159513 · Cited by 5
“19 Heterogeneity was calculated with Cochran Q test using the package of “ RVAideMemoire ” , following null hypothesis and low heterogeneity was de fi ned as “ I2 < 50%, p > .1 ” .”
MethodNiu, L., Xiao, L., Cai, Q.-H., Wu, Y.-Y., Hu, S.-Y., Guo, S.-X., Tian, Y.-L., & Wang, Q.-R. (2023) · Integrative Medicine Research · doi:10.1016/j.imr.2023.100944 · Cited by 5
Terao, I., Yokoi, A., & Fukushima, H. (2023) · Journal of Psychopharmacology · doi:10.1177/02698811231200020 · Cited by 6
“In addition, the certainty of evidence contributing to network estimates of the primary outcomes was assessed using the Grading of Recommendations Assessment, Development, and Evaluation framework (Salanti et al., 2014).”
MethodChang, H., Yeung, T. C., Yang, X., Gao, J., Wu, X., & Wang, C. C. (2023) · Human Fertility · doi:10.1080/14647273.2023.2266619 · Cited by 6
“The grades of recommendation, assessment, development, and evaluation (GRADE) system were adopted to grade the quality of the evidence into four levels: high, middle, low, and very low (Salanti et al., 2014).”
MethodParry‐Smith, W., Gallos, I. D., Williams, H., Widmer, M., Angolkar, M., Tobı́as, A., Price, M. J., Alfirević, Ž., Weeks, A. D., Hofmeyr, G. J., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012754 · Cited by 5
Athanasiou, A., Veroniki, A. A., Efthimiou, O., Kalliala, I. E. J., Naci, H., Bowden, S. J., Paraskevaidi, M., Martin‐Hirsch, P. L., Bennett, P. R., Paraskevaidis, E., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028008 · Cited by 6
Self-citationVeroniki, A. A., Straus, S. E., Ashoor, H. M., Hamid, J. S., Yu, C. H. Y., & Tricco, A. C. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-010160 · Cited by 6
Bjerring, P. N., Morgan, M. Y., Vilstrup, H., Nielsen, S. M., Christensen, R., & Gluud, L. L. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013241 · Cited by 5
Barry, G. S., & Uppal, V. (2020) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-020-01819-w · Cited by 5
Zhang, Y., Zhang, H., Zhang, Y., Yang, Z., Wang, L., Pan, W., Dai, R., Ju, Q., Ren, D., Bao, S., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281469 · Cited by 4
Wigle, A., Béliveau, A., Salanti, G., Rücker, G., Schwarzer, G., Mavridis, D., & Nikolakopoulou, A. (2025) · Statistics in Medicine · doi:10.1002/sim.70176 · Cited by 4
“POTH is a quantitative approach, which sets it apart from some previously suggested approaches to quantifying uncertainty in treatment hierarchies [8, 9].”
InfluentialSelf-citationZhao, P., He, Y., Li, M., Wang, J., Wang, R., & Cui, X. (2025) · PM&R · doi:10.1002/pmrj.13417 · Cited by 6
Wang, Z., Li, J., Wang, L., & Wei, Q. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116665 · Cited by 5
Ma, Y., Yan, R., Li, Y., Li, D., Sun, X., Chen, T., & Liu, X. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1640858 · Cited by 5
Wang, Z., Yang, H., Li, S., Lu, C., Yuan, Y., Bai, Y., Su, J., Li, Y., Wang, T., & Xu, Z. (2025) · Maturitas · doi:10.1016/j.maturitas.2025.108713 · Cited by 5
Yang, Y., Xu, Y., Tang, H., Hu, Y., & Bai, F. (2025) · BMC Anesthesiology · doi:10.1186/s12871-025-03395-2 · Cited by 6
“…setting), this study adopted a head-to-head design and focused on comparing the relative efficacy of multiple interventions rather than evaluating their absolute effects [47, 48] The hip joint receives its primary innervation from the lower lumbar plexus (L2-L4) and the upper sacral plexus (L4-S1).”
Lai, Y., Pan, Q., Wang, S., Luo, Q., Huang, Z., Wei, M., Jiang, Z., & Yi, W. (2025) · Frontiers in Pharmacology · doi:10.3389/fphar.2025.1683857 · Cited by 6
Yi, Y., Yang, J., Chen, Y., Liu, H., & Deng, Y. (2025) · Annals of Medicine · doi:10.1080/07853890.2025.2597063 · Cited by 4
“CINeMA assesses the credibility of outcomes through six dimensions: within-study bias, reporting bias, indirectness, imprecision, heterogeneity and inconsistency [49].”
Efendi, D., Kang, Y.-N., Natalia, R., Milanti, A., Marsubrin, P. M. T., Kustanti, C. Y., & Chen, K. (2026) · Systematic Reviews · doi:10.1186/s13643-025-03043-3 · Cited by 5
Schreiber, S., Danese, S., Colombel, J., Hisamatsu, T., Irving, P. M., Park, H. K., Kim, D. H., Lee, Y. N., & Hanauer, S. B. (2026) · Advances in Therapy · doi:10.1007/s12325-025-03447-6 · Cited by 7
Tien, C. H., Chiu, M. C., Shen, Y. L., Ko, Y. C., & Lee, J. J. (2026) · Scientific Reports · doi:10.1038/s41598-026-40038-z · Cited by 4
“Our NMA expands the literature in several directions.”
InfluentialTan, H., Yan, Z., Shao, C., & Lin, Y. (2026) · Frontiers in Physiology · doi:10.3389/fphys.2026.1769079 · Cited by 6
“If the gap between the ranked values of each intervention is small, indicating poor stability, the intervention will be reduced by one level (Salanti et al., 2014).”
Xin, X., Guo, Y., Wang, M., Hu, Z., Chen, J., & Xie, J. (2026) · Journal of the American Heart Association · doi:10.1161/jaha.125.044003 · Cited by 8
“39”
Lu, W., Yang, Z., Qi, Z., & Yang, X. (2022) · Chinese Journal of Plastic and Reconstructive Surgery · doi:10.1016/j.cjprs.2022.08.007 · Cited by 4
Feng, Z., Wu, S., Hu, H. X., Long, H., Zhou, L., & Shen, M. (2024) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-024-05129-5 · Cited by 4
“Comparison-adjusted funnel plots and Egger’s test were used to evaluate the publication bias in NMAs [35].”
MethodShi, J., Li, P., Wu, Y., Li, J., Zhang, Y., & Xiao, B. (2024) · JMIR Research Protocols · doi:10.2196/55870 · Cited by 4
“Additionally, if possible, we will use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [26] to assess the certainty of evidence.”
MethodZhang, X.-N., Liang, Z., & Li, M. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1494368 · Cited by 9
Zhou, K., Ren, Y., Zang, L., & Zhou, Z. (2025) · Climacteric · doi:10.1080/13697137.2025.2509866 · Cited by 4
“Studies with no high risk and ≤3 unclear risks were rated as high quality, articles with one high or unclear risk ≥4 were rated as moderate quality and all other cases were rated as low quality [27,28].”
Huang, S., & Zhang, L. (2025) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2025.105996 · Cited by 7
Michael, H. J., Pham, H. D., Li, L. S. K., Semciw, A. I., Scholes, M. J., Hannington, M. C., & King, M. (2025) · Physiotherapy Research International · doi:10.1002/pri.70105 · Cited by 7
“This heterogeneity in criteria used for knee OA diagnosis may influence the characteristics in the sample population (Bedson and Croft 2008), contributing to the indirectness observed in the results (Salanti et al. 2014).”
Liu, W., Yin, M., & Li, H. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1623679 · Cited by 4
Interventions for treating urinary incontinence in older women: a network meta-analysis
Vesentini, G., O’Connor, N., Berre, M. L., Nabhan, A. F., Wagg, A., Wallace, S. A., & Dumoulin, C. L. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015376.pub2 · Cited by 7
EFFICACY OF HEMOSTATIC AGENTS IN ENDODONTIC SURGERY: A SYSTEMATIC REVIEW AND NETWORK META-ANALYSIS
Khater, A. G. A., Al‐Hamed, F. S., Safwat, E. M., HAMOUDA, M. M. A., Shehata, M., & Scarano, A. (2021) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2021.101540 · Cited by 4
Pascual‐Morena, C., Cavero‐Redondo, I., Álvarez‐Bueno, C., Martínez‐Hortelano, J. A., Reina‐Gutiérrez, S., Saz‐Lara, A., de Arenas‐Arroyo, S. N., & Martínez‐Vizcaíno, V. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.796009 · Cited by 4
“Grading the Quality of Evidence We used the Grading of Recommendations, Assessment, Development and Evaluation tool to assess the quality of evidence and make recommendations (29, 30).”
MethodManagement of Adult Unruptured Brain Arteriovenous Malformations: An Updated Network Meta-Analysis
Dmytriw, A. A., Ku, J. C., Ghozy, S. A., Grewal, S. S., Cancelliere, N. M., Azzam, A. Y., Regenhardt, R. W., Rabinov, J. D., Stapleton, C. J., Patel, K., et al. (2023) · Neurointervention · doi:10.5469/neuroint.2023.00171 · Cited by 5
Chen, Z., Zhou, Y., Liu, X., Zhao, W., Zhao, G., Zheng, J., Zhang, G., Wang, Y., Zhang, S., & Song, Y. (2023) · Optics & Laser Technology · doi:10.1016/j.optlastec.2023.109870 · Cited by 4
Papola, D., Karyotaki, E., Purgato, M., Sijbrandij, M., Tedeschi, F., Cuijpers, P. J., Orestis, E., Furukawa, T. A., Patel, V., & Barbui, C. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-077037 · Cited by 4
“32 Stepped care as well as collaborative care interventions will be excluded as they preclude homogeneity in intervention and components administration within trial arms.”
BackgroundYang, T.-L., Lin, C. S., Ho, C., Huang, T., Wu, Y., Jhou, H., Chen, P., & Lee, C. (2023) · Life · doi:10.3390/life13122259 · Cited by 4
“SUCRA transforms the mean rank into a simple value, which offers a hierarchy of the treatments that considers both the location and variance of all relative treatment effects [24–26].”
BackgroundSun, X., Chen, J., Zou, Y., Lei, J., & Liu, W. (2024) · Medicine · doi:10.1097/md.0000000000038014 · Cited by 4
“[25]”
Liang, S., He, Y., Chen, X., Wu, M., & Li, K. (2024) · International Journal of Dental Hygiene · doi:10.1111/idh.12878 · Cited by 4
Some reflections on the evolution of meta‐analysis
Altman, D. G. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1156 · Cited by 5
Pan, J., Hu, X., Yin, H., Zhang, C., & Yan, Z. (2021) · Medicine · doi:10.1097/md.0000000000025597 · Cited by 4
“The area under the curve increases as the SUCRA value increases, indicating that the intervention is more effective.([19])”
BackgroundMethodLees, S., Dicker, M., Ku, J. E., Chaganti, V., Mew-Sum, M., Wang, N., Smith, A., Oldmeadow, C. J., Goon, W. L., Bevan, M., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-051509 · Cited by 4
“50 This characterises the quality of a body of evidence contributing to a NMA based on study limitations, imprecision, heterogeneity or inconsistency, indirectness and publication bias.”
Okano, H., Sakuraya, M., Masuyama, T., Kimata, S., & Hokari, S. (2022) · JA Clinical Reports · doi:10.1186/s40981-022-00525-4 · Cited by 5
“The transitivity assumption was evaluated by comparing the distribution of potential effect modifiers across treatment comparisons [22].”
MethodBoczar, K. E., Shin, S., Bezzina, K. A., Geejo, A., Pearson, A., Shahab, S., Fehlmann, C. A., Visintini, S. M., Beanlands, R. S., & Wells, G. A. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-062702 · Cited by 5
Lü, J., Lin, R., Zhang, C., Zhang, M., Wang, Y., & Zhang, Y. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058788 · Cited by 7
Biologics for chronic severe asthma: a network meta-analysis
Crossingham, I. R., Richardson, R., Hinks, T., Spencer, S., Couillard, S., Maynard-Paquette, A.-C., Thomassen, D., & Howell, I. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015411 · Cited by 4
Oh, S., & Kim, E. Y. (2023) · Clinical and Experimental Medicine · doi:10.1007/s10238-023-01041-0 · Cited by 4
Meng, X., Zhu, X., Li, B., Liu, J., Zhao, J., Wang, H., Feng, Q., & Su, Y. (2023) · Experimental and Therapeutic Medicine · doi:10.3892/etm.2023.11971 · Cited by 4
Qiu, M., Li, C., Sun, T., & Ruan, Q. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1133585 · Cited by 4
“The quality of evidence contributing to each network was evaluated by the Grading of Recommendations Assessment, Development and Evaluation (GRADE), to determine the evidence quality of the two outcomes according to study limitations, imprecise data, inconsistency, indirectness and potential publication bias (10).”
MethodWeng, Y., Miao, B., Hong, D., Zhang, M., Wang, B., Zhao, Q. D., & Wang, H. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-072960 · Cited by 7
“68 The direct and indirect estimates rating will consider to be rated down due to the following reasons: risk of bias, inconsistency, indirectness, imprecision and publication bias.”
BackgroundMethodA gentle introduction to network meta-analysis for orthodontists
Tu, Y., Hsu, J.-Y., Chang, Y.-H., Zheng, K.-W., & Pandis, N. (2024) · Seminars in Orthodontics · doi:10.1053/j.sodo.2024.01.009 · Cited by 4
Kim, Y., Vakula, M. N., Bolton, D. A. E., Dakin, C. J., Thompson, B. J., Slocum, T. A., Teramoto, M., & Bressel, E. (2022) · Vide Leaf, Hyderabad eBooks · doi:10.37247/pans2ed.2.22.5 · Cited by 4
Kitagawa, T., Denda, T., Okuyama, W., Miyachi, R., & Nakamura, K. (2024) · Cureus · doi:10.7759/cureus.57661 · Cited by 5
“The identification of publication bias is reported to be more complex in NMAs [22], although the use of comparison-adjusted funnel plots [40] or regression methods and selection models [41] is encouraged.”
MethodWang, W., Liu, F., Hu, T., & Wang, C. (2018) · Medicine · doi:10.1097/md.0000000000011447 · Cited by 4
Zheng, W., Li, M., Yu, H., Xie, F., Yan, Q., Peng, Z., Huang, H.-T., Liao, H., & Liu, X. (2019) · Medicine · doi:10.1097/md.0000000000016299 · Cited by 4
“The complete assessment procedures are as follows:([7])”
Xiong, C., Li, Y., Zeng, Y., Wei, H., Zhuang, G., Li, L., Zhao, L.-H., Li, C., Qin, E., & Fu, J. (2019) · Medicine · doi:10.1097/md.0000000000017002 · Cited by 4
“The quality of evidence for the main outcomes will be assessed with the GRADE (the Grading of Recommendations Assessment, Development and Evaluation) approach.([50]) These 5 items will be investigated, including limitations of study design, inconsistencies, inaccuracies, indirection, and publication bias.”
MethodTwiss, M., Hilfiker, R., Hinrichs, T., de Bruin, E. D., & Rogan, S. (2019) · Physical Therapy Reviews · doi:10.1080/10833196.2019.1664082 · Cited by 4
He, S., Zou, Y., Li, J., Liu, J., Zhao, L., Yang, H., Su, Z., & Ye, H. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-033917 · Cited by 4
Luo, H., Zheng, W., Yan, Q., Wang, B., Ye, B., Deng, S., He, F., Yang, X., & Wang, X. (2020) · Medicine · doi:10.1097/md.0000000000020500 · Cited by 4
“2) 144/142 194/92 D A (1)(2)(3)(4) Huang YN 2012([23]) ≥60 III–IV 60–77 (67.”
Forget, P., & Vermeersch, M. (2020) · British Journal of Pain · doi:10.1177/2049463720945289 · Cited by 4
Systemic treatments for eczema: a network meta-analysis
Sawangjit, R., Dilokthornsakul, P., Lloyd-Lavery, A., Chua, S. K. J., Lai, N. M., Dellavalle, R. P., & Chaiyakunapruk, N. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013206 · Cited by 5
Statins in adult patients with HIV: Protocol for a systematic review and network meta-analysis
Roever, L., Resende, E. S., Diniz, A. L. D., Penha‐Silva, N., O’Connell, J. L., Gomes, P. F. S., Zanetti, H. R., Roerver-Borges, A. S., Veloso, F. C., Fidale, T. M., et al. (2018) · Medicine · doi:10.1097/md.0000000000010116 · Cited by 6
Chen, L., Ferreira, P. H., Beckenkamp, P. R., & Ferreira, M. L. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024752 · Cited by 5
“Grading the evidence The Grading of Recommendations, Assessment, Development and Evaluations framework will be used to evaluate the quality of evidence.(51) The tool includes five domains, which are study limitations, indirectness, inconsistency, imprecision and publication bias.”
MethodAlternative reactive support surfaces (non-foam or air-filled) for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013623 · Cited by 4
Substitutes of Prescription Medicines - A Review of Concerns Relevant to Doctors and Patients
Kairi, J. K. (2017) · JOURNAL OF CLINICAL AND DIAGNOSTIC RESEARCH · doi:10.7860/jcdr/2017/28552.10439 · Cited by 4
Beds and mattresses for treating pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., & McInnes, E. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013624 · Cited by 4
Deconstructing the Network Meta-analysis
Westafer, L. M., & Schriger, D. L. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.02.019 · Cited by 4
Xu, L., Xiang, L., Pan, L., Xue, P., Li, J., He, Y., Liu, H., Hu, Y., & Zheng, B. (2025) · Frontiers in Pediatrics · doi:10.3389/fped.2025.1547308 · Cited by 3
Li, X.-J., Xia, H., Yu, G.-H., Hu, Y., & Zhu, F. (2025) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2025.102004 · Cited by 4
Terao, I., Tsuge, T., Sato, T., Endo, K., Kotake, K., Banno, M., & Kataoka, Y. (2025) · Psychiatry and Clinical Neurosciences Reports · doi:10.1002/pcn5.70136 · Cited by 10
Zhong, Z., Guo, W., Gu, Q., Pang, T., Li, L., & Hao, A. (2025) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-025-08876-6 · Cited by 8
“In accordance with the GRADE system for assessing evidence quality, the strength of the study’s findings may be downgraded based on five factors: (1) Risk of (5) Publication bias: asymmetry in the funnel plot [69, 70].”
Rujirawan, T., Osiri, S., & Chotvorrarak, K. (2025) · International Endodontic Journal · doi:10.1111/iej.70007 · Cited by 5
“Consequently, we were unable to globally test for inconsistency (referred to as in-coherence in CINeMA) within the network (Salanti et al. 2014).”
Cheng, H., Zhou, Z. Y., Wang, H., & Hang, X. (2025) · Psycho-Oncology · doi:10.1002/pon.70222 · Cited by 3
“In addition, we utilized the GRADE (Grading, Assessment, Development, and Evaluation of Recommendations) system [27], according to which the reliability (or quality of evidence) was categorized as high, moderate, low, and very low.”
Peng, P., Wei, W., Lin, L.-F., Liu, Y., Yan, X.-Y., Tan, Y.-T., Zhang, Z., Zhang, W., & Huang, Y.-X. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10063-x · Cited by 4
“Of note, we used these techniques because it is often difficult to make judgements about publication bias in NMA (Salanti et al., 2014).”
Tan, Y., Liu, Y., Koga, A., Yuan, Y., Yu, H., & Liu, J. (2025) · Healthcare · doi:10.3390/healthcare13172132 · Cited by 5
“NMA not only provides estimates of relative effectiveness among treatments that have not been directly compared but also ranks interventions based on their efficacy and safety profiles [21].”
Wen, S., Yan, Y., Shao, J., Xie, H., Wang, M., Dou, Y., Liu, D., Xiao, Y., & Ma, X. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03751-3 · Cited by 7
“For duloxetine and vortiox-etine, the curve depicted an inverted U-shaped.”
Shen, X., Chen, L., Shen, L., Chen, S., Mu, S., Chen, L., Wang, S., & Li, Z. (2025) · Frontiers in Neurology · doi:10.3389/fneur.2025.1634369 · Cited by 4
Zhang, X.-N., Liang, Z., & Li, M. (2025) · Supportive Care in Cancer · doi:10.1007/s00520-025-09971-z · Cited by 5
“Following this, we graded the certainty of evidence for NMA outcomes utilizing the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework [48].”
Elgendy, M. S., Rifai, M. K., Taha, A. M., Rajab, I. M., Maged, A., Elgamasy, M. A., Taha, H. I. A., Abuelazm, M. T., Mohan, B. P., & Adler, D. G. (2025) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2025.12.268 · Cited by 5
Cheng, H.-Y., Liang, C.-W., Liu, Y., Chang, C., Chang, W.-L., Cheng, T.-A., Chuang, H., Hsu, Y.-C., & Lin, C. (2025) · Global Spine Journal · doi:10.1177/21925682251408367 · Cited by 3
“The certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, following the method outlined by Salanti et al. 23 GRADE classi fi es evidence quality into 4 levels: very low, low, moderate, and high.”
Liu, T., Chen, Z., & Gao, K. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1732663 · Cited by 5
Rakhsha, S. S., Shahinfar, H., Ebrahimi-Mousavi, S., Kosari, A., Aali, Y., Mirzababaei, A., Hosseini, S. A., Jadidi, P., Shafiee, G., Mansourzadeh, M. J., et al. (2026) · Diabetes Obesity and Metabolism · doi:10.1111/dom.70957 · Cited by 4
Kishi, T., Sakuma, K., & Iwata, N. (2022) · Pharmacopsychiatry · doi:10.1055/a-1860-2793 · Cited by 3
“Finally, we incorpo - rated the results into the Confidence in Network Meta-Analysis (CINeMA) application, an adaptation of the Grading of Recommen - dations Assessment, Development, and Evaluation approach, to assess the credibility of the findings of each network meta-analysis [18–20].”
de Souza, M. B., Mascarenhas, R. O., Maia, L. B., Fonseca, L. S., Silva, H. J., de Zoete, R. M. J., McAuley, J. H., Henschke, N., & Oliveira, V. C. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274406 · Cited by 4
Yan, X., Peng, P., & Liu, Y. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.108062 · Cited by 9
Sandau, N., Aagaard, T. V., Hróbjartsson, A., Harris, I. A., & Brorson, S. (2024) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-023-07119-w · Cited by 4
“This is concerning, given that the results obtained from network meta-analyses are invalid if the transitivity assumption is violated [54, 55].”
BackgroundZha, F., Wen, Q., Zhou, M., Shan, L., & Wang, Y. (2024) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000002571 · Cited by 3
Mahardawi, B., Thet, P. H., Kaboosaya, B., & Pimkhaokham, A. (2025) · Oral and Maxillofacial Surgery · doi:10.1007/s10006-025-01330-w · Cited by 3
“The quality of evidence was evaluated for both the mixed and indirect evidence, in order to estimate the overall quality of evidence from this study [26].”
Vichara-Anont, I., Lumkul, L., Phinyo, P., Wongsa, C., & Thongngarm, T. (2025) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2025.02.018 · Cited by 3
Salam, A., Dhurjati, R., Vidyasagar, K., Kaistha, P., Esam, H., Haghdoost, F., Pant, R., Kumar, A., Kanukula, R., Thode, R., et al. (2025) · JMIR Research Protocols · doi:10.2196/65205 · Cited by 4
“The approach suggested by Salanti et al [20] and Adriani Nikolakopoulou et al [21] using the CiNeMA tool will be used to assess confidence in the evidence from network meta-analyses.”
Zhu, B., Zhu, D., Xue, X., Yang, H., & Zhang, S. (2025) · Journal of Medical Internet Research · doi:10.2196/56227 · Cited by 6
“Furthermore, the quality level of this meta-analysis was evaluated according to GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach [44,45].”
Immunomodulators and immunosuppressants for myasthenia gravis: a network meta-analysis
Varela, L. B., Meza, N., Liquitay, C. M. E., Menai, S. D., Franco, J. V. A., & Garegnani, L. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016208 · Cited by 5
Liu, F., Wang, C., Hu, T., & Wang, W. (2018) · Medicine · doi:10.1097/md.0000000000010397 · Cited by 3
Examining the use of network meta-analysis in pharmacy services research: A systematic review
Teoh, K. W., Khan, T. M., Chaiyakunapruk, N., & Lee, S. W. H. (2019) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2019.06.015 · Cited by 3
Price, J. W. (2021) · Journal of Osteopathic Medicine · doi:10.1515/jom-2020-0268 · Cited by 3
“Heterogeneity was not a consideration for this model because each direct comparison in the network had only a single study; inconsistency was not assessed because the model had no closed loops [25].”
BackgroundMatterne, U., Tischer, C., Wang, J., Knüttel, H., Genuneit, J., Perkin, M. R., & Apfelbacher, C. (2021) · F1000Research · doi:10.12688/f1000research.51490.2 · Cited by 3
Li, J., Wu, Q., Ng, E. H. Y., Mol, B. W., Wu, X. K., & Wang, C. C. (2021) · Engineering · doi:10.1016/j.eng.2021.07.009 · Cited by 4
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.