Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 201–300 of 999
Wong, H.-Y., Pilling, R. J., Young, B. W. M., Owolabi, A. A., Onwochei, D. N., & Desai, N. (2021) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2021.110274 · Cited by 64
A network meta-analysis of psychosocial interventions for refugees and asylum seekers with PTSD
Turrini, G., Tedeschi, F., Cuijpers, P. J., Giovane, C. D., Kip, A., Morina, N., Nosè, M., Ostuzzi, G., Purgato, M., Ricciardi, C., et al. (2021) · BMJ Global Health · doi:10.1136/bmjgh-2021-005029 · Cited by 76
Machine perfusion in liver transplantation
Tingle, S. J., Dobbins, J. J., Thompson, E. R., Figueiredo, R., Mahendran, B. S., Pandanaboyana, S., & Wilson, C. H. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014685.pub2 · Cited by 112
Li, X., Zhang, L., Gu, S., Sun, J., Qin, Z., Yue, J., Zhong, Y., Ding, N., & Gao, R. (2018) · Medicine · doi:10.1097/md.0000000000012819 · Cited by 62
“After all the outcomes were evaluated, a summary of findings table was created using the GRADE system,([29]) following the 4step approach to rate the quality of evidence in each of the direct and mixed estimates based on methods developed by the GRADE working group.([30,31])”
MethodChen, J.-J., Zeng, B., Wu, C., Stubbs, B., Carvalho, A. F., Brunoni, A. R., Su, K., Tu, Y., Wu, Y., Chen, T., et al. (2020) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2020.1497 · Cited by 66
Wattanayingcharoenchai, R., Rattanasiri, S., Charakorn, C., Attia, J., & Thakkinstian, A. (2021) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16366 · Cited by 58
Comparative Efficacy of Cabozantinib and Regorafenib for Advanced Hepatocellular Carcinoma
Kelley, R. K., Mollon, P., Blanc, J., Daniele, B., Yau, T. C. C., Cheng, A., Valcheva, V., Marteau, F., Guerra, I., & Abou‐Alfa, G. K. (2020) · Advances in Therapy · doi:10.1007/s12325-020-01378-y · Cited by 59
Gulea, C., Zakeri, R., Alderman, V., Morgan, A., Ross, J., & Quint, J. K. (2021) · Respiratory Research · doi:10.1186/s12931-021-01661-8 · Cited by 70
“Small study effects were explored by conducting comparison-adjusted funnel plots [20] and publication bias was assessed by Egger’s test among comparisons of beta-blockers and placebo.”
MethodWheat, C. L., Ko, C. W., Clark‐Snustad, K., Grembowski, D. E., Thornton, T. A., & Devine, B. (2017) · BMC Gastroenterology · doi:10.1186/s12876-017-0602-0 · Cited by 58
“The strength of evidence was assessed utilizing The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach specifically designed for network meta-analysis [47].”
BackgroundMethodForget, P., Borovac, J. A., Thackeray, E. M., & Pace, N. L. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003006.pub4 · Cited by 60
“'Summary of findings' table and GRADE Methods for evaluating the quality of evidence of a NMA have been proposed by Puhan and colleagues (Puhan 2014), and Salanti and colleagues (Salanti 2014).”
BackgroundMethodFranco, J. V. A., Jung, J. H., Imamura, M., Borofsky, M. S., Omar, M. I., Liquitay, C. M. E., Young, S., Golzarian, J., Veroniki, A. A., Garegnani, L., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013656.pub2 · Cited by 81
Cohen, A. T., Hill, N. R., Luo, X., Masseria, C., Abariga, S. A., & Ashaye, A. O. (2018) · International Journal of Cardiology · doi:10.1016/j.ijcard.2018.06.114 · Cited by 57
Association of dengue disease severity and blood group: A systematic review and meta‐analysis
Hashan, M. R., Ghozy, S. A., El‐Qushayri, A. E., Pial, R. H., Hossain, M. A., & Kibria, G. M. A. (2020) · Reviews in Medical Virology · doi:10.1002/rmv.2147 · Cited by 63
“To assess the risk of bias and small-study effects, comparison-adjusted funnel plots were developed.(38) Furthermore, funnel plot asymmetry was assessed with three different tests; Egger's”
MethodShi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., McInnes, E., Goh, E. L., & Norman, G. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013761.pub2 · Cited by 98
Seow, W., Dudi-Venkata, N. N., Bedrikovetski, S., Kroon, H. M., & Sammour, T. (2022) · Techniques in Coloproctology · doi:10.1007/s10151-022-02739-1 · Cited by 67
“Thirdly, SUCRA rankings do not account for surgeon’s familiarity and higher cost of novel surgical techniques [72].”
BackgroundPaschos, P., Katsoula, A., Salanti, G., Γιουλεμέ, Ό., Athanasiadou, E. A., & Τσάπας, Α. (2018) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.15005 · Cited by 56
“For continuous outcomes, we compared the estimated τ(2) with a reference median value for τ(2) for “pharmacological intervention vs placebo” comparisons provided by Rhodes et al(17) Low heterogeneity was considered when the estimated τ(2) was smaller than the reference median value for τ(2) for the specific type of ou…”
MethodSelf-citationPease, A., Lo, C., Earnest, A., Kiriakova, V., Liew, D., & Zoungas, S. (2020) · Diabetes Care · doi:10.2337/dc19-1785 · Cited by 70
Tolley, K. H., Hutchinson, M., You, X., Wang, P., Sperling, B., Taneja, A., Siddiqui, M. K., & Kinter, E. T. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127960 · Cited by 59
“This study was therefore identified for further examination to assess the inconsistency of evidence [44].”
BackgroundHou, L., Wang, Q., Pan, B., Li, R., Li, Y., He, J., Qin, T., Cao, L., Zhang, N., Cao, C., et al. (2023) · Diabetes/Metabolism Research and Reviews · doi:10.1002/dmrr.3591 · Cited by 56
“Combined aerobic and resistance exercise was more powerful in reducing HbA1c than aerobic exercise alone, in line with the data published by Schwingshackl et al.38 and our previous NMA.14 The effects of aerobic and resistance exercises on glycaemic control in patients with type 2 diabetes are complementary.”
MethodInfluentialBradley, A., & Meer, R. V. D. (2019) · Scientific Reports · doi:10.1038/s41598-019-40951-6 · Cited by 62
A Gentle Introduction to Bayesian Network Meta-Analysis Using an Automated R Package
Liu, Y., Béliveau, A., Wei, Y., Chen, M. Y., Record-Lemon, R., Kuo, P., Pritchard, E., Tang, X., & Chen, G. (2022) · Multivariate Behavioral Research · doi:10.1080/00273171.2022.2115965 · Cited by 64
“However, each has its own limitations and may give poor results when unexplained heterogeneity increases, and the number of studies decreases despite some suggestions that selection models may work better than other methods (Mavridis et al., 2014; Salanti et al., 2014).”
ResultA brief note on the random-effects meta-analysis model and its relationship to other models
McKenzie, J. E., & Veroniki, A. A. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111492 · Cited by 69
Rausa, E., Kelly, M. E., Asti, E., Aiolfi, A., Bonitta, G., & Bonavina, L. (2018) · Surgical Endoscopy · doi:10.1007/s00464-018-6592-3 · Cited by 57
“The confidence in the estimates was assessed using CINeMA and emerged to range from moderate to very low primarily due to the quality of the included studies [76].”
MethodAskar, H., Krois, J., Göstemeyer, G., & Schwendicke, F. (2020) · Journal of Dentistry · doi:10.1016/j.jdent.2020.103541 · Cited by 63
Rogozińska, E., Daru, J., Nicolaides, M., Amezcua‐Prieto, C., Robinson, S. E., Wang, R., Godolphin, P. J., Saborido, C. M., Zamora, J., Khan, K. S., et al. (2021) · The Lancet Haematology · doi:10.1016/s2352-3026(21)00137-x · Cited by 75
“The network map was generated for both efficacy outcomes and examined for its connectivity (presence of closed loops).(22)”
Silverberg, J. I., Hong, H., Thyssen, J. P., Calimlim, B., Joshi, A. D., Teixeira, H. D., Collins, E. B., Crowell, M. M., Johnson, S. J., & Armstrong, A. W. (2022) · Dermatology and Therapy · doi:10.1007/s13555-022-00721-1 · Cited by 61
“Additionally, NMAs are susceptible to the methodological quality of included studies, reporting biases, and choices of study eligibility criteria [50].”
BackgroundKelley, G. A., Kelley, K. S., & Pate, R. R. (2016) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12228 · Cited by 68
“to 2 levels for each domain and up to a maximum of 3 levels across all 5 domains (58).”
BackgroundMethodInfluentialFilippini, G., Giovane, C. D., Clerico, M., Beiki, O., Mattoscio, M., Piazza, F., Fredrikson, S., Tramacere, I., Scalfari, A., & Salanti, G. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012200.pub2 · Cited by 65
Self-citationCui, Y., Li, G., Cao, R., Luan, L., & Kla, K. M. (2019) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2019.06.028 · Cited by 65
O'Connor, A. M. C., Hu, D., Totton, S. C., Scott, N. M., Winder, C. B., Wang, B., Wang, C., Glanville, J., Wood, H., White, B. J., et al. (2019) · Animal Health Research Reviews · doi:10.1017/s1466252319000288 · Cited by 55
Samara, M. T., Huhn, M., Chiocchia, V., Schneider‐Thoma, J., Wiegand, M. H., Salanti, G., & Leucht, S. (2020) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13201 · Cited by 70
“We assessed the confidence in estimates of the primary outcomes with the Confidence in Network MetaAnalysis (CINeMA) framework and web application (38-40).”
MethodSelf-citationPapola, D., Ostuzzi, G., Tedeschi, F., Gastaldon, C., Purgato, M., Giovane, C. D., Pompoli, A., Pauley, D., Karyotaki, E., Sijbrandij, M., et al. (2021) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.148 · Cited by 71
Ghozy, S. A., Kacimi, S. E. O., Azzam, A. Y., Farahat, R. A., Abdelaal, A., Kallmes, K. M., Adusumilli, G., Heit, J. J., Kadirvel, R., & Kallmes, D. F. (2022) · Journal of NeuroInterventional Surgery · doi:10.1136/neurintsurg-2021-018436 · Cited by 58
Enomoto, K., Adachi, T., Fujino, H., Kugo, M., Tatsumi, S., & Sasaki, J. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101693 · Cited by 56
Jakob, T., Tesfamariam, Y. M., Macherey‐Meyer, S., Kuhr, K., Adams, A., Monsef, I., Heidenreich, A., & Skoetz, N. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013020.pub2 · Cited by 66
Babatunde, O. O., Ensor, J., Littlewood, C., Chesterton, L. S., Jordan, J. L., Wynne‐Jones, G., Roddy, E., Foster, N. E., & van der Windt, D. (2021) · Therapeutic Advances in Musculoskeletal Disease · doi:10.1177/1759720x211037530 · Cited by 60
Network Meta-Analysis of Initial Antithrombotic Regimens After Left Atrial Appendage Occlusion
Carvalho, P. E. P., Gewehr, D. M., Miyawaki, I. A., Nogueira, A., Felix, N. S., Garot, P., Darmon, A., Mazzone, P., Preda, A., Nascimento, B. R., et al. (2023) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2023.08.010 · Cited by 53
Liang, Z., Zhang, M., Wang, C., Hao, F., Yu, Y., Tian, S., & Yuan, Y. (2024) · Sports Medicine · doi:10.1007/s40279-024-02057-6 · Cited by 63
“The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) [21] method was used to assess the quality of evidence for ranking outcomes of different exercise interventions based on NMAs.”
Burr, N. E., Lummis, K., Sood, R., Kane, J. S., & Subramanian, V. (2016) · The Lancet. Gastroenterology & hepatology · doi:10.1016/s2468-1253(16)30162-5 · Cited by 60
Tsiami, A. P., Chaimani, A., Mavridis, D., Siskou, M., Assimakopoulos, E., & Sotiriadis, A. (2016) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.15900 · Cited by 58
Self-citationTäger, T., Atar, D., Agewall, S., Katus, H. A., Grundtvig, M., Cleland, J. G. F., Clark, A. L., Fröhlich, H., & Frankenstein, L. (2020) · Heart Failure Reviews · doi:10.1007/s10741-020-09954-8 · Cited by 63
Fountain, D. M., Bryant, A., Barone, D. G., Waqar, M., Hart, M. G., Bulbeck, H., Kernohan, A., Watts, C., & Jenkinson, M. D. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013630.pub2 · Cited by 67
“See: Summary of findings 1 iMRI image-guided surgery compared to standard surgery for high-grade glioma; Summary of findings 2 5-ALA image-guided surgery compared to standard surgery for high-grade glioma; Summary of findings 3 Neuronavigation image-guided surgery compared to standard surgery for high-grade glioma Due…”
BackgroundMethodInfluentialDeng, L., Sun, X., Qiu, S., Xiong, Y., Li, Y., Wang, L., Wei, Q., Wang, D., & Liu, M. (2017) · Scientific Reports · doi:10.1038/s41598-017-16663-0 · Cited by 59
Chung, E. Y. M., Palmer, S. C., Saglimbene, V. M., Craig, J. C., Tonelli, M. A., & Strippoli, G. F. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010590.pub3 · Cited by 79
Zeng, W., Liu, Z., Dai, H., Yan, M., Luo, H. L., Ke, M., & Cai, X. (2017) · BMC Ophthalmology · doi:10.1186/s12886-017-0601-5 · Cited by 49
“This study was performed according to the Preferred Reporting Items for Systematic Reviews and Metaanalyses (PRISMA) statement and the Cochrane Collaboration recommendations [28, 29].”
MethodRausa, E., Zappa, M. A., Kelly, M. E., Turati, L., Russo, A., Aiolfi, A., Bonitta, G., & Sgroi, L. G. (2019) · Techniques in Coloproctology · doi:10.1007/s10151-019-02034-6 · Cited by 48
In network meta-analysis, most of the information comes from indirect evidence: empirical study
Papakonstantinou, T., Nikolakopoulou, A., Egger, M., & Salanti, G. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.04.009 · Cited by 63
“risk of bias, whose influence on the results could be critical (7,8).”
BackgroundSelf-citationEstimating the contribution of studies in network meta-analysis: paths, flows and streams
Papakonstantinou, T., Nikolakopoulou, A., Rücker, G., Chaimani, A., Schwarzer, G., Egger, M., & Salanti, G. (2018) · F1000Research · doi:10.12688/f1000research.14770.1 · Cited by 49
Self-citationAssessing Confidence in the Results of Network Meta-Analysis (Cinema)
Nikolakopoulou, A., Higgins, J. P., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., & Salanti, G. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/597047 · Cited by 49
“CINeMA is based on 53 the GRADE framework, with several conceptual and semantic differences (5).”
BackgroundMethodSelf-citationLu, J., Xia, Q., Lu, Y., Liu, Z., Zhou, P., Hu, H., & Wang, S. (2020) · Cancer Medicine · doi:10.1002/cam4.3513 · Cited by 47
Sensitivity of Treatment Recommendations to Bias in Network Meta-Analysis
Phillippo, D., Dias, S., Ades, A. E., Didelez, V., & Welton, N. J. (2017) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12341 · Cited by 50
“Recently, two methods to extend the GRADE framework to NMA have been proposed (Puhan et al. , 2014; Salanti et al. , 2014).”
InfluentialCurrent treatment of ocular toxoplasmosis in immunocompetent patients: a network meta-analysis
Zhang, Y., Lin, X., & Lü, F. (2018) · Acta Tropica · doi:10.1016/j.actatropica.2018.04.026 · Cited by 52
Leite, R. G. O. F., Banzato, L. R., Galendi, J. S. C., Mendes, A. L., Bolfi, F., Veroniki, A. A., Thabane, L., & Nogueira, V. D. S. N. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034481 · Cited by 62
Liu, Q., Ni, W., Zhang, L., Zhao, M., Bai, X., Zhang, S., Ding, Y., Yin, H., & Chen, L. (2023) · Ageing Research Reviews · doi:10.1016/j.arr.2023.102071 · Cited by 55
Kang, D. H., Cho, K. S., Ham, W. S., Lee, H., Kwon, J. K., Choi, Y. D., & Lee, J. Y. (2016) · PLoS ONE · doi:10.1371/journal.pone.0158661 · Cited by 51
“Similar to heterogeneity, inconsistency is caused by effect modifiers and specifically by an imbalance in the distribution of effect modifiers in the direct and indirect evidence from the network meta-analysis [37,38].”
Bonovas, S., Nikolopoulos, G. K., Lytras, T., Fiorino, G., Peyrin‐Biroulet, L., & Danese, S. (2017) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.13456 · Cited by 54
de Geus, J. L., da Costa, J. K. N., Wambier, L. M., Maran, B. M., Loguércio, A. D., & Reis, A. (2019) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2019.09.002 · Cited by 47
Ryan, O. K., Ryan, É. J., Creavin, B., Rausa, E., Kelly, M. E., Petrelli, F., Bonitta, G., Kennelly, R. P., Hanly, A. M., Martin, S., et al. (2020) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2020.06.037 · Cited by 51
Aiolfi, A., Lombardo, F., Matsushima, K., Sozzi, A., Cavalli, M., Panizzo, V., Bonitta, G., & Bona, D. (2021) · Surgery · doi:10.1016/j.surg.2021.04.014 · Cited by 49
Kim, Y., Vakula, M. N., Bolton, D. A. E., Dakin, C. J., Thompson, B. J., Slocum, T. A., Teramoto, M., & Bressel, E. (2022) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.764826 · Cited by 61
“Consistency, which is the most important assumption underlying a NMA and indicates agreement between direct and indirect estimates in the network (Salanti et al., 2014), was checked using the node-splitting analysis.”
Interventions for myopia control in children: a living systematic review and network meta-analysis
Lawrenson, J. G., Huntjens, B., Virgili, G., Ng, S., Dhakal, R., Downie, L. E., Verkicharla, P. K., Kernohan, A., Li, T., & Walline, J. J. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014758.pub3 · Cited by 121
Jung, S., Jang, E. J., Nam, S. W., Kwon, H. H., Im, S. G., Kim, D., Cho, S., Kim, D., & Sung, Y. (2018) · Modern Rheumatology · doi:10.1080/14397595.2018.1439694 · Cited by 46
“Second, unlike previous network meta-analyses, we restricted our study population and interventions to knee OA and oral interventions, and separated the analysis of pain into WOMAC pain and pain VAS to minimize possible concern about the indirectness due to different population, intervention, and outcome in network me…”
BackgroundResultEfficacy and Safety of Antibiotic Therapy in Early Cutaneous Lyme Borreliosis
Torbahn, G., Hofmann, H., Rücker, G., Bischoff, K., Freitag, M., Dersch, R. S., Fingerle, V., Motschall, E., Meerpohl, J. J., & Schmucker, C. M. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.3186 · Cited by 47
“The certainty of evidence for the outcomes response to treatment (≥12 months following start of treatment) and any treatment-related adverse event was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for NMA, which is associated with specific comparisons, including…”
Wu, S., Gao, L., Cipriani, A., Huang, Y., Yang, Z., Yang, J., Yu, S., Zhang, Y., Chai, S., Zhang, Z., et al. (2018) · Diabetes Obesity and Metabolism · doi:10.1111/dom.13613 · Cited by 48
“Additionally, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework was used to assess the quality of evidence contributing to each network estimate, which characterizes the quality of a body of evidence on the basis of study limitations, imprecision, inconsistency, indirectness and…”
MethodMortada, H. H., AlNojaidi, T. F., AlRabah, R., Almohammadi, Y., AlKhashan, R., & Aljaaly, H. A. (2022) · The Breast Journal · doi:10.1155/2022/7857158 · Cited by 54
“05 indicated statistical significance [20].”
Liao, C.-D., Huang, S.-W., Chen, H.-C., Huang, M.-H., Liou, T.-H., & Lin, C.-L. (2024) · Nutrients · doi:10.3390/nu16070941 · Cited by 45
“The downgrading level for the study limitation was based on the amounts of contribution of each direct estimate and corresponding risk of bias to the NMA estimates [86,87].”
BackgroundCortese, S., Adamo, N., Mohr-Jensen, C., Hayes, A. J., Bhatti, S., Carucci, S., Del Giovane, C., Atkinson, L. Z., Banaschewski, T., Simonoff, E., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013967 · Cited by 45
Jung, A., Eschke, R.-C., Struss, J., Taucher, W. K., & Luedtke, K. (2022) · Cephalalgia · doi:10.1177/03331024221082073 · Cited by 45
Reina‐Varona, Á., Madroñero‐Miguel, B., Fierro‐Marrero, J., Paris‐Alemany, A., & Touche, R. L. (2024) · Headache The Journal of Head and Face Pain · doi:10.1111/head.14696 · Cited by 57
“26 The study design domain was down-graded by 1 point if studies presented some concerns and by 2 points if they presented a high risk of bias.”
Comparative effects of different whole grains and brans on blood lipid: a network meta-analysis
Hui, S., Liu, K., Lang, H., Liu, Y., Wang, X., Zhu, X., Doucette, S. P., Yi, L., & Mi, M. (2018) · European Journal of Nutrition · doi:10.1007/s00394-018-1827-6 · Cited by 49
Chatzakis, C., Goulis, D. G., Mareti, E., Eleftheriades, M., Zavlanos, A., Dinas, K., & Sotiriadis, A. (2019) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2019.107924 · Cited by 45
Interventions for treating iron deficiency anaemia in inflammatory bowel disease
Gordon, M. A., Sinopoulou, V., Iheozor‐Ejiofor, Z., Iqbal, T. H., Allen, P. B., Hoque, S., Engineer, J., & Akobeng, A. K. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013529.pub2 · Cited by 50
Siciliano, M. A., Caridà, G., Ciliberto, D., D’Apolito, M., Pelaia, C., Caracciolo, D., Riillo, C., Correale, P., Galvano, A., Russo, A., et al. (2022) · ESMO Open · doi:10.1016/j.esmoop.2022.100465 · Cited by 44
“We subsequently carried out a Bayesian NMA(14) using STATA software (StataCorp, version 17 College Station, TX) implemented by a graphical tool and the ‘mvmeta’ package.(15,16) In NMA, direct and indirect comparisons between different treatments are possible by logical inference, allowing to rank (from best to worst)…”
MethodDesai, N., Ratnayake, G., Onwochei, D. N., El‐Boghdadly, K., & Ahmad, I. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.05.025 · Cited by 46
Mechanical bowel preparation and antibiotics in elective colorectal surgery: network meta-analysis
Tan, J., Ryan, É. J., Davey, M. G., McHugh, F. T., Creavin, B., Whelan, M. C., Kelly, M. E., Neary, P. C., Kavanagh, D. O., & O’Riordan, J. M. (2023) · BJS Open · doi:10.1093/bjsopen/zrad040 · Cited by 51
Kitridis, D., Tsikopoulos, K., Bisbinas, I., Papaioannidou, P., & Givissis, P. (2019) · The American Journal of Sports Medicine · doi:10.1177/0363546518823337 · Cited by 48
“This assessment was achieved qualitatively and quantitatively after visually inspecting the comparison-adjusted funnel plots for potential asymmetry and following the execution of the Egger statistical test, correspondingly.(21,60)”
MethodAlternating pressure (active) air surfaces for preventing pressure ulcers
Shi, C., Dumville, J. C., Cullum, N. A., Rhodes, S., Jammali‐Blasi, A., & McInnes, E. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013620.pub2 · Cited by 70
Nóbrega, D. B., Naqvi, S. A., Dufour, S., Deardon, R., Kastelic, J. P., Buck, J. D., & Barkema, H. W. (2020) · Journal of Dairy Science · doi:10.3168/jds.2020-18365 · Cited by 44
“In this scenario, the assumption of transitivity (no systematic differences among comparisons of same treatments) becomes crucial (Salanti et al., 2014).”
BackgroundRadadiya, D. K., Devani, K., Brahmbhatt, B. S., & Reddy, C. M. (2021) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002035 · Cited by 55
Wang, G., Yang, Q., Wu, L., & Jiang, G. (2021) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000005521 · Cited by 44
Lü, Y., Zhu, H., Wang, Q., Tian, C., Lai, H., Hou, L., Liu, Y., Gao, Y., Liu, M., Yang, F., et al. (2022) · Sleep Medicine · doi:10.1016/j.sleep.2022.03.012 · Cited by 49
Common bile duct stones management: A network meta-analysis
Mohseni, S., Bass, G. A., Forssten, M. P., Martínez-Casas, I., Martin, M. J., Davis, K. A., Haut, E. R., Sugrue, M., Kurihara, H., Sarani, B., et al. (2022) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000003755 · Cited by 43
“The comparison-adjusted funnel plots were used to visualize the risk of publication bias in the network meta-analyses; the numerical Egger's test for publication bias was also used where possible.(12) A ranking probability plot was used to rank the treatments (a high value indicates that the treatment might be better).”
MethodXie, P., Luo, M., Deng, X., Fan, J., & Xiong, L. (2023) · Nutrients · doi:10.3390/nu15173856 · Cited by 50
Cheng, L., Yan, J., Han, S., Chen, Q., Chen, M., Jiang, H., & Lü, J. (2019) · Critical Care · doi:10.1186/s13054-019-2427-4 · Cited by 51
Zhao, L., Wang, L., Jia, X., Hu, X., Pang, P., Zhao, S., Wang, Y., Wang, J., Zhang, Y., & Lyu, Z. (2020) · Frontiers in Oncology · doi:10.3389/fonc.2020.540238 · Cited by 45
“’ Moreover, we used the GRADE framework to develop and present summaries of evidence (26).”
MethodLin, L., Li, T., Sun, M., Liang, Q., Ma, Y., Wang, F., Duan, J., & Sun, Z. (2022) · Environmental Research · doi:10.1016/j.envres.2022.112785 · Cited by 51
Zheng, J., Zhang, H., Shi, J., Li, X., Zhang, J., Zhang, K., Gao, Y., He, J., Dai, J., & Wang, J. (2024) · Science of The Total Environment · doi:10.1016/j.scitotenv.2023.168589 · Cited by 42
Antoniou, S. A., Mavridis, D., Hajibandeh, S., Hajibandeh, S., Antoniou, G. A., Gorter, R. R., Tenhagen, M., Koutras, C., Pointner, R., Chalkiadakis, G., et al. (2017) · Surgery · doi:10.1016/j.surg.2017.07.013 · Cited by 46
Créquit, P., Chaimani, A., Yavchitz, A., Attiche, N., Cadranel, J. L., Trinquart, L., & Ravaud, P. (2017) · BMC Medicine · doi:10.1186/s12916-017-0954-x · Cited by 43
“[23], which extends the GRADE system into NMA.”
BackgroundSelf-citationNetwork meta-analysis: users’ guide for pediatricians
Khalifah, R. A. A., Flórez, I. D., Guyatt, G. H., & Thabane, L. (2018) · BMC Pediatrics · doi:10.1186/s12887-018-1132-9 · Cited by 45
“Although, the scope of the research question can vary from narrow to broad, it is essential that for any paired comparison within the NMA, it is plausible that we will, for each outcome of interest, observe similar effects across all patient populations being addressed [30, 31].”
BackgroundEfficacy and Safety of Stents in ST-Segment Elevation Myocardial Infarction
Chichareon, P., Modolo, R., Collet, C. A., Tenekecioğlu, E., Vink, M. A., Oh, P. C., Ahn, J., Musto, C., de la Llera, L. S. D., Cho, Y., et al. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2019.09.038 · Cited by 47
Tseng, P., Yang, C., Su, K., Chen, T., Wu, Y., Tu, Y., Lin, P., Stubbs, B., Carvalho, A. F., Matsuoka, Y., et al. (2020) · Journal of Pineal Research · doi:10.1111/jpi.12663 · Cited by 45
Rijs, K., Mercier, F. J., Lucas, D. N., Rossaint, R., Klimek, M., & Heesen, M. A. (2020) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000001371 · Cited by 59
Smith, R. C., Leucht, S., & Davis, J. M. (2019) · Psychopharmacology · doi:10.1007/s00213-018-5133-z · Cited by 41
Lattanzi, S., Trinka, E., Russo, E., Giovane, C. D., Matricardi, S., Meletti, S., Striano, P., Damavandi, P. T., Silvestrini, M., & Brigo, F. (2023) · Drugs · doi:10.1007/s40265-023-01936-y · Cited by 48
“Network meta-analyses within a frequentist framework were then performed assuming equal heterogeneity across all comparisons [14, 15].”
MethodIheozor‐Ejiofor, Z., Gordon, M. A., Clegg, A. J., Freeman, S. C., Gjuladin-Hellon, T., MacDonald, J. K., & Akobeng, A. K. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013210.pub2 · Cited by 41
Aldin, A., Besiroglu, B., Adams, A., Monsef, I., Piechotta, V., Tomlinson, E., Hornbach, C., Dressen, N., Goldkuhle, M., Maisch, P., et al. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013798.pub2 · Cited by 51
Tramacere, I., Giovane, C. D., Salanti, G., D’Amico, R., Pacchetti, I., & Filippini, G. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011381 · Cited by 43
Self-citationCao, J., Liu, J., Zhang, X., & Wang, Z. (2022) · Journal of Vascular Surgery Venous and Lymphatic Disorders · doi:10.1016/j.jvsv.2022.08.004 · Cited by 41
Kishi, T., Ikuta, T., Citrome, L., Sakuma, K., Hatano, M., Hamanaka, S., Nishii, Y., & Iwata, N. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03439-8 · Cited by 49
“Finally, the results were integrated into the Con fi dence in Network Meta-Analysis application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach, to assess the credibility of the fi ndings of each of the network meta-analyses [34 – 36].”
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.