Evaluating the Quality of Evidence from a Network Meta-Analysis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, to assess the credibility of the findings of each network meta-analysis, the findings were incorporated into the Confidence in Network Meta-Analysis (CINeMA) application, which is an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation approach [19–21].”
From Antidepressants for the treatment of adults with major depressive disorder in the maintenance phase: a systematic review and network meta-analysis (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We rated the credibility of the evidence included in the network meta-analysis using CINeMA33 (appendix pp 16–17).”
From Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Additionally, we assessed the certainty of evidence contributing to network estimates of the main outcomes with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.15”
The cited source says what the claim attributes to it.
In the citing paper
“Where there appeared to be a bene fi t of probiotics over placebo in analyses containing more than 1 RCT, we assessed the level of certainty of evidence according to GRADE criteria. 37”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“However, due to a combination of 1 or more of heterogeneity between studies, possible publication bias, risk of bias of studies, and modest ef fi cacy, the level of certainty of evidence would not be graded as anything more than low or very low, according to GRADE criteria, 37 in almost all of our analyses.”
From Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis (2023) · cites this paper as [37]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Salanti, G., Ades, A. E., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.016 · Cited by 3,766 · cited 2×
“The ranking of the treatments should be done using probabilistic methods, for example using ‘rankograms’ or the surface under the cumulative ranking curve (SUCRA), which take into account the estimated effect sizes and their accompanying uncertainty [8].”
MethodSelf-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · cited 2×
“In the GRADE approach, randomized trials are evaluated according to generation of the allocation sequence, concealment of the allocation sequence, blinding, incomplete accounting of participants and outcome events, selective outcome reporting bias and other limitations [15].”
MethodA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“A particularly convenient way to do this is to look at predictive intervals for the effect in a new study of each comparison [14;25;26].”
BackgroundSelf-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“Approaches that focus on evaluating the involvement of a particular comparison in network inconsistency (such as the node-splitting approach or simple comparisons of direct and indirect estimates [22;23]) are more relevant to making judgements about (pairwise) network meta-analysis effect estimates than are global ass…”
BackgroundSelf-citationGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“The contribution plot can be obtained in STATA (see [14]).”
BackgroundMethodSelf-citationGRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,166
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098 · cited 1×
“The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
BackgroundSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Cited by
Items 301–400 of 999
Breast Cancer Chemoprevention: A Network Meta-Analysis of Randomized Controlled Trials
Mocellin, S., Pilati, P., Briarava, M., & Nitti, D. (2015) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djv318 · Cited by 42
Sridharan, K., & Sivaramakrishnan, G. (2018) · Journal of Clinical Neuroscience · doi:10.1016/j.jocn.2018.02.011 · Cited by 40
A generalized pairwise modelling framework for network meta-analysis
Doi, S. A. R., & Barendregt, J. J. (2018) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000140 · Cited by 42
Mavridis, D., Porcher, R., Nikolakopoulou, A., Salanti, G., & Ravaud, P. (2019) · Biometrical Journal · doi:10.1002/bimj.201900026 · Cited by 41
“A natural further step is to extend the CINeMA framework to evaluate an obtained treatment hierarchy as initially described in Salanti et al. (2014).”
BackgroundSelf-citationThom, H. H. Z., Norman, G., Welton, N. J., Crosbie, E. J., Blazeby, J. M., & Dumville, J. C. (2020) · Surgical Infections · doi:10.1089/sur.2019.318 · Cited by 48
“The random effects heterogeneity variance, interpreted in line with empiric ranges in the literature, was used as an across-contrast assessment of heterogeneity [27,30].”
Phinyo, P., Koompawichit, P., Nochaiwong, S., Tovanabutra, N., Chiewchanvit, S., & Chuamanochan, M. (2020) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2020.08.055 · Cited by 44
Chen, J., Ho, M., Lee, K., Song, Y., Fang, Y., Goldstein, B. A., He, W., Irony, T., Jiang, Q., van der Laan, M. J., et al. (2021) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2021.1883474 · Cited by 45
“The Working Group was organized into two workstreams.”
BackgroundInfluentialAcupuncture and Related Therapies for Obesity: A Network Meta‐Analysis
Zhang, Y., Li, J., Mo, G., Liu, J., Yang, H., Chen, X., Liu, H., Cai, T., Zhang, X., Tian, X., et al. (2018) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2018/9569685 · Cited by 43
“By using the technique of network meta-analysis (NMA), both direct and indirect randomized data can be analyzed, and recommended rankings of different treatments can be provided [15, 16].”
MethodCheng, A. S. K., Wang, X., Niu, N., Liang, M., & Zeng, Y. (2022) · Neuropsychology Review · doi:10.1007/s11065-021-09532-1 · Cited by 46
“Similarity, also referred to as transitivity, requires study characteristics such as study population and intervention types or duration to be sufficiently similar in different sets of clinical trials; heterogeneity is the extent to which treatment effect varies in direct comparisons (a difference which needs to be mi…”
BackgroundZadeh, E. M., Ziafat, K., Yazdani, K., Kamel, M. M., Wong, J. S. H., Modabbernia, A., Blanken, P., Verthein, U., Schütz, C. G., Jang, K. L., et al. (2021) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2021.1887202 · Cited by 45
“(53), based on the GRADE guidelines for the assessment of the quality of evidence and overall rating of confidence in the estimates (54).”
BackgroundHayashi, M., Yamamoto, N., Kuroda, N., Kano, K., Miura, T., Kamimura, Y., & Shiroshita, A. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2024.01.024 · Cited by 49
Terao, I., Honyashiki, M., & Inoue, T. (2022) · Ageing Research Reviews · doi:10.1016/j.arr.2022.101709 · Cited by 42
Kobeissi, H., Ghozy, S. A., Turfe, B., Bilgin, C., Kadirvel, R., Kallmes, D. F., Brinjikji, W., & Rabinstein, A. A. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1102463 · Cited by 45
“Whenever ten or more comparisons were pooled for an outcome, comparisonadjusted funnel plots were built to examine the risk of bias and smallstudy effects (19).”
BackgroundKishi, T., Yoshimura, R., Sakuma, K., Okuya, M., & Iwata, N. (2020) · Neuropsychopharmacology Reports · doi:10.1002/npr2.12137 · Cited by 37
Estimating the contribution of studies in network meta-analysis: paths, flows and streams
Papakonstantinou, T., Nikolakopoulou, A., Rücker, G., Chaimani, A., Schwarzer, G., Egger, M., & Salanti, G. (2018) · F1000Research · doi:10.12688/f1000research.14770.3 · Cited by 36
Self-citationInterventions for displaced radial head fractures: network meta-analysis of randomized trials
Vannabouathong, C., Akhter, S., Athwal, G. S., Moro, J. K., & Bhandari, M. (2019) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2018.10.019 · Cited by 39
Schwingshackl, L., Schwarzer, G., Rücker, G., & Meerpohl, J. J. (2019) · Advances in Nutrition · doi:10.1093/advances/nmz036 · Cited by 37
“CINeMA is an adaptation of the GRADE approach and considers the following items: RoB, indirectness, imprecision, heterogeneity, publication bias, and inconsistency (102).”
Gerger, H., Werner, C. P., Gaab, J., & Cuijpers, P. J. (2021) · Psychological Medicine · doi:10.1017/s0033291721000143 · Cited by 50
Comparative Efficacy of Bronchiolitis Interventions in Acute Care: A Network Meta-analysis
Elliott, S. A., Gaudet, L., Fernandes, R. M., Vandermeer, B., Freedman, S. B., Johnson, D. W., Plint, A. C., Klassen, T. P., Allain, D., & Hartling, L. (2021) · PEDIATRICS · doi:10.1542/peds.2020-040816 · Cited by 42
Qin, Z., Zhang, C., Guo, J., Kwong, J. S., Li, X., Pang, R., Doiron, R. C., Nickel, J. C., & Wu, J. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101457 · Cited by 44
From living systematic reviews to meta-analytical research domains
Cuijpers, P. J., Miguel, C., Papola, D., Harrer, M., & Karyotaki, E. (2022) · Evidence-Based Mental Health · doi:10.1136/ebmental-2022-300509 · Cited by 40
Chaw, S. H., Lo, Y. L., Goh, S. L., Cheong, C. C., Tan, W. K., Loh, P. S., Wong, L. F., & Shariffuddin, I. I. (2021) · Obesity Surgery · doi:10.1007/s11695-021-05564-x · Cited by 37
Impact of no‐tillage on soil quality and crop yield in Asia: A meta‐analysis
Hashimi, R., Kaneko, N., & Komatsuzaki, M. (2022) · Land Degradation and Development · doi:10.1002/ldr.4512 · Cited by 42
“I2 showed variability due to study heterogeneity, where I2 < 50% is accounted for low heterogeneity, 50%–75% is counted as moderate heterogeneity, and >75% is accounted for high heterogene- ity (Salanti et al., 2014).”
BackgroundWu, S., Cipriani, A., Yang, Z., Yang, J., Cai, T., Xu, Y., Quan, X., Zhang, Y., Chai, S., Sun, F., et al. (2018) · Expert Opinion on Drug Safety · doi:10.1080/14740338.2018.1424826 · Cited by 36
“Additionally, the GRADE (The Grading of Recommendations Assessment, Development, and Evaluation) framework was used to assess the quality of evidence contributing to each network estimate, which characterizes the quality of a body of evidence on the basis of the study limitations, imprecision, inconsistency, indirectn…”
MethodBest practices for MRI systematic reviews and meta‐analyses
McGrath, T. A., Bossuyt, P. M., Cronin, P., Salameh, J., Kraaijpoel, N., Schieda, N., & McInnes, M. D. F. (2018) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26198 · Cited by 38
Iannone, A., Ruospo, M., Palmer, S. C., Principi, M., Barone, M., Leo, A. D., & Strippoli, G. F. (2019) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.15493 · Cited by 37
Zhou, X., Yao, S., Dong, P., Chen, B., Xu, Z., & Wang, H. X. (2020) · Critical Care · doi:10.1186/s13054-020-03090-3 · Cited by 40
Franco, J. V. A., Jung, J. H., Imamura, M., Borofsky, M. S., Omar, M. I., Liquitay, C. M. E., Young, S., Veroniki, A. A., Garegnani, L., & Dahm, P. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013656 · Cited by 37
A systematic review and network meta‐analysis of interventions for subjective cognitive decline
Roheger, M., Hennersdorf, X.-S., Riemann, S., Flöel, A., & Meinzer, M. (2021) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1002/trc2.12180 · Cited by 44
Täger, T., Frankenstein, L., Atar, D., Agewall, S., Frey, N., Grundtvig, M., Clark, A. L., Cleland, J. G. F., & Fröhlich, H. (2021) · Clinical Research in Cardiology · doi:10.1007/s00392-021-01913-z · Cited by 39
Murphy, D., O’Donnell, J. P., Ryan, É. J., O’Neill, B. L., Boland, M. R., Lowery, A., Kerin, M. J., & Inerney, N. M. M. (2022) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000009984 · Cited by 41
Li, S., Li, Y., Xu, C., Tao, S., Sun, H., Wang, Y., Wang, Y., Li, S., & Ma, X. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02295-2 · Cited by 48
“To resolve this, we used the comparison-adjusted funnel plot method to assess the risk of publication bias 38 .”
Clinical studies in restorative dentistry: Design, conduct, analysis
Schwendicke, F., & Opdam, N. J. M. (2017) · Dental Materials · doi:10.1016/j.dental.2017.09.009 · Cited by 46
Andreou, A., Watson, D. I., Mavridis, D., Francis, N. K., & Antoniou, S. A. (2019) · Surgical Endoscopy · doi:10.1007/s00464-019-07208-9 · Cited by 41
Schwingshackl, L., Schlesinger, S., Devleesschauwer, B., HOFFMANN, G. E., Bechthold, A., Schwedhelm, C., Iqbal, K., Knüppel, S., & Boeing, H. (2018) · Proceedings of The Nutrition Society · doi:10.1017/s0029665118000125 · Cited by 38
“The system is transparent and applicable to any network structure((49)).”
Sargeant, J. M., Brennan, M. L., & O'Connor, A. M. C. (2022) · Frontiers in Veterinary Science · doi:10.3389/fvets.2022.960957 · Cited by 52
“The interested reader is referred to further details on the components used in evaluating evidence for CATs (4), for SR-MA using the GRADE approach (52), for network meta-analysis (53) and for clinical practice guidelines (8, 44).”
MethodHerranz‐Gómez, A., Cuenca‐Martínez, F., Suso‐Martí, L., Varangot‐Reille, C., Prades-Monfort, M., Calatayud, J., & Casaña, J. (2023) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2023.01.008 · Cited by 45
Qin, R., Li, Z., Huang, K., Luan, T., Miao, B., Gong, L., Liu, W., & Chen, L. (2025) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2025.105941 · Cited by 46
Ungprasert, P., Thongprayoon, C., & Davis, J. M. (2015) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2015.09.004 · Cited by 37
“This uneven distribution can occur even if only RCTs are included, as randomization does not hold across the entire set of included trials, since patients are not randomized to different trials [42].”
BackgroundLang, C. I., Wolfien, M., Langenbach, A., Müller, P., Olaf Wolkenhauer, A., Yavari, A., İnce, H., Steinhoff, G., Krause, B. J., David, R., et al. (2017) · Cellular Physiology and Biochemistry · doi:10.1159/000477324 · Cited by 34
Percutaneous patent foramen ovale closure for secondary stroke prevention: Network meta-analysis
Tsivgoulis, G. K., Katsanos, A. H., Mavridis, D., Frogoudaki, A., Vrettou, A., Ikonomidis, I., Parissis, J., Deftereos, S. G., Karapanayiotides, T., Palaiodimou, L., et al. (2018) · Neurology · doi:10.1212/wnl.0000000000005739 · Cited by 39
Gao, Y., Hou, L., Lu, C., Wang, Q., Pan, B., Wang, Q., Tian, J., & Ge, L. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.01186 · Cited by 35
“We rated the certainty (quality) of evidence using the GRADE approach that classified evidence as high, moderate, low, and very low certainty (Salanti et al., 2014).”
MethodChen, W., Tsai, S. H. L., Goyal, A., Fu, T., Lin, T., & Bydon, M. (2020) · European Spine Journal · doi:10.1007/s00586-020-06636-9 · Cited by 36
Chang, P., Chen, K.-H., Jhou, H., Chen, P., Huang, C., Lee, C., & Chang, T. (2021) · Scientific Reports · doi:10.1038/s41598-021-94716-1 · Cited by 37
Giovane, C. D., Boncoraglio, G. B., Bertù, L., Banzi, R., & Tramacere, I. (2021) · BMC Neurology · doi:10.1186/s12883-021-02341-2 · Cited by 43
López, P., Rech, A., Πετροπούλου, Μ., Newton, R. U., Taaffe, D. R., Galvao, D. A., Turella, D. J. P., Freitas, S. R., & Radaelli, R. (2022) · The Journals of Gerontology Series A · doi:10.1093/gerona/glac230 · Cited by 41
“The several domains considered for downgrading the certainty of evidence were study limitations (ie, based on the risk of bias assessment), indirectness (ie, based on transitivity assumption), inconsistency (ie, based on inconsistency assumption), imprecision (ie, uncertainty around the pairwise estimate with confiden…”
BackgroundMethodTian, S., Liang, Z., Qui, F., Yu, Y., Wang, C., Zhang, M., & Wang, X. (2024) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2024.06.031 · Cited by 42
Barbato, L., Selvaggi, F., Kalemaj, Z., Buti, J., Bendinelli, E., Marca, M. L., & Cairo, F. (2020) · Clinical Oral Investigations · doi:10.1007/s00784-020-03229-0 · Cited by 36
“Prior to conducting a NMA, the assumptions of transitivity and consistency between comparisons were examined [24, 25].”
MethodPratt, M., Wieland, L. S., Ahmadzai, N., Butler, C., Wolfe, D. M., Pussagoda, K., Skidmore, B., Veroniki, A. A., Rios, P. D. L., Tricco, A. C., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01328-3 · Cited by 32
Kampolis, C. F., Mermiri, M., Mavrovounis, G., Koutsoukou, A., Loukeri, A. A., & Pantazopoulos, I. N. (2021) · Journal of Critical Care · doi:10.1016/j.jcrc.2021.11.010 · Cited by 39
Willems, L. H., Maas, D. P. M. S. M., Kramers, K., Reijnen, M. M. P. J., Riksen, N. P., Cate, H. T., van der Vijver-Coppen, R. J., de Borst, G. J., Mees, B., Zeebregts, C. J., et al. (2022) · Drugs · doi:10.1007/s40265-022-01756-6 · Cited by 38
“We explored the potential for publication bias by visual inspection of the comparison-adjusted funnel plots [32, 33].”
MethodInterventions for myopia control in children: a living systematic review and network meta-analysis
Lawrenson, J. G., Dhakal, R., Verkicharla, P. K., Shah, R., Huntjens, B., Downie, L. E., Kernohan, A., Li, T., Virgili, G., & Walline, J. J. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014758 · Cited by 35
“• Credibility (e.g. change in the confidence in one or more CINeMA domains (Confidence In Network Meta-analysis; Salanti 2014)).”
InfluentialWhich Exercise Approaches Work for Relieving Cancer-Related Fatigue? A Network Meta-analysis
Dong, B., Qi, Y., Lü, L., Liu, T., Wang, S., Zhang, Y., Yuan, Y., Cheng, H., Chen, Q., Fang, Q., et al. (2023) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2023.11251 · Cited by 41
Hu, H., Wu, C., Kwok, J. Y. Y., Ho, M., Chau, P. H., Lok, K. Y. W., & Choi, E. P. H. (2023) · American Journal of Nephrology · doi:10.1159/000532109 · Cited by 47
“The quality of evidence was evaluated using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) for pairwise meta-analysis, as well as the ranking of NMA [31].”
MethodMahardawi, B., Jiaranuchart, S., Arunjaroensuk, S., Dhanesuan, K., Mattheos, N., & Pimkhaokham, A. (2024) · Clinical Oral Implants Research · doi:10.1111/clr.14357 · Cited by 32
Stahl‐Pehe, A., Mashhadi, N. S., Wirth, M., Schlesinger, S., Kuß, O., Holl, R. W., Bächle, C., Warz, K.-D., Bürger-Büsing, J., Spörkel, O., et al. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103190 · Cited by 42
“18 We further present the network estimates of mean differences along with their 95% CIs and 95% predictive intervals (95% PrIs) in predictive interval plots to indicate the interval within which the relative treatment effect of a future study is expected to lie.”
Zanjir, M., Sgro, A., Lighvan, N. L., Yarascavitch, C., Shah, P. S., da Costa, B. R., & Azarpazhooh, A. (2020) · Journal of Endodontics · doi:10.1016/j.joen.2020.07.002 · Cited by 42
Bae, J.-H., & Lee, H. (2020) · Clinical Nutrition · doi:10.1016/j.clnu.2020.11.006 · Cited by 36
Howle, R., Ng, S. C., Wong, H.-Y., Onwochei, D. N., & Desai, N. (2021) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-021-02128-6 · Cited by 36
Zhou, X., Cipriani, A., Zhang, Y., Cuijpers, P., Hetrick, S. E., Weisz, J. R., Pu, J., Giovane, C. D., Furukawa, T. A., Barth, J., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-016608 · Cited by 31
“GRADE quality assessment We will also assess the quality of evidence contributing to primary outcomes with the GRADE framework, which characterises the quality of a body of evidence on the basis of the study limitations, imprecision, heterogeneity or inconsistency, indirectness and publication bias.(55) The starting p…”
MethodSelf-citationBahji, A., Ermacora, D., Stephenson, C., Hawken, E. R., & Vazquez, G. (2021) · The Canadian Journal of Psychiatry · doi:10.1177/0706743720970857 · Cited by 31
Grant, S., Azhar, G., Han, E., Booth, M., Motala, A., Larkin, J., & Hempel, S. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003822 · Cited by 31
Comparative effects of different types of cardioplegia in cardiac surgery: A network meta-analysis
Tan, J., Bi, S., Li, J., Gu, J., Wang, Y., Xiong, J., Yu, X., & Du, L. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.996744 · Cited by 39
“The transitivity assumption (14) was evaluated by comparing the distribution of the following potential effect modifiers across studies: publication frequency, median cross time, median CPB time, sex, and mean age.”
MethodPhinyo, P., Krikeerati, T., Vichara-Anont, I., & Thongngarm, T. (2023) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2023.11.007 · Cited by 35
Xie, W., Lu, D., Liu, S., Li, J., & Li, R. (2024) · Preventive Medicine · doi:10.1016/j.ypmed.2024.107955 · Cited by 40
Maqsood, M. H., Yong, C. M., Rao, S. V., Cohen, M. G., Pancholy, S. B., & Bangalore, S. (2024) · Circulation Cardiovascular Interventions · doi:10.1161/circinterventions.124.014186 · Cited by 39
Risk of Bias in Network Meta-Analysis (RoB NMA) tool
Lunny, C., Higgins, J. P. T., White, I. R., Dias, S., Hutton, B., Wright, J. M., Veroniki, A. A., Whiting, P. F., & Tricco, A. C. (2025) · BMJ · doi:10.1136/bmj-2024-079839 · Cited by 41
“4 In practice, transitivity requires that the sets of studies used to obtain the indirect comparison are sufficiently similar in characteristics that may modify the intervention effects.”
Self-citationLiu, F., Wang, W., Wang, C., & Peng, X. (2018) · Medicine · doi:10.1097/md.0000000000010016 · Cited by 37
Chaimani, A., Mavridis, D., Salanti, G., Higgins, J. P. T., & White, I. R. (2018) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1801800310 · Cited by 31
Self-citationZheng, Q., Yang, H., Liu, W., Sun, W., Zhao, Q. C., Zhang, X., Jin, H.-N., & Sun, L. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030919 · Cited by 32
“(2 trials, 18 patients), CH (4, 112), CTX (16, 487), CsA”
Benenati, S., Crimi, G., Canale, C., Pescetelli, F., Marzo, V. D., Vergallo, R., Galli, M., Bona, R. D., Canepa, M., Ameri, P., et al. (2020) · European Heart Journal - Cardiovascular Pharmacotherapy · doi:10.1093/ehjcvp/pvaa127 · Cited by 33
Xie, P.-W., Yan, J., Ye, L., Wang, C., Li, Y., Chen, Y., & Li, G. (2021) · Surgical Endoscopy · doi:10.1007/s00464-021-08386-1 · Cited by 34
Progestogens for preventing miscarriage: a network meta-analysis
Devall, A., Papadopoulou, A., Haas, D. M., Price, M. J., Coomarasamy, A., & Gallos, I. D. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013792 · Cited by 32
Systematic review with network meta-analysis: dual therapy for high-risk bleeding peptic ulcers
Shi, K., Shen, Z., Zhu, G., Meng, F.-S., Gu, M., & Ji, F. (2017) · BMC Gastroenterology · doi:10.1186/s12876-017-0610-0 · Cited by 31
Paracetamol, NSAIDS and opioid analgesics for chronic low back pain: a network meta-analysis
Bagg, M. K., McLachlan, A. J., Maher, C. G., Kamper, S. J., Williams, C. M., Henschke, N., Wand, B. M., Moseley, G. L., Hübscher, M., O'Connell, N. E., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013045 · Cited by 34
Radaelli, R., Rech, A., Molinari, T., Markarian, A. M., Πετροπούλου, Μ., Granacher, U., Hortobágyi, T., & López, P. (2024) · Sports Medicine · doi:10.1007/s40279-024-02123-z · Cited by 46
“The certainty of evidence for the network of interventions was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for network meta-analysis [42].”
Zaccardi, F., Dhalwani, N. N., Dales, J., Mani, H., Khunti, K., Davies, M. J., & Webb, D. R. (2018) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.13185 · Cited by 30
“SUs were the most common second agent combined with metformin (20 RCTs; 46.5%), followed by TZDs (10; 23.2%), basal or rapid-acting insulin (6; 13.9%), DPP-4 inhibitors (3; 7.0%), GLP-1RAs (2; 4.6%), or SGLT-2 inhibitors (2; 4.6%); references of included studies are reported in Appendix S1.”
MethodInfluentialEfficacy and tolerability of pharmacotherapy for post-stroke depression: a network meta-analysis
Deng, L., Qiu, S., Yang, Y., Wang, L., Li, Y., Lin, J., Wei, Q., Yang, L., Wang, D., & Liu, M. (2018) · Oncotarget · doi:10.18632/oncotarget.23891 · Cited by 35
“In this approach, direct evidence from RCTs starts at high quality and can be downgraded based on risk of bias, imprecision, indirectness, inconsistency (or heterogeneity) and publication bias to levels of moderate, low and relatively low quality [54].”
BackgroundTsikopoulos, K., Mavridis, D., Georgiannos, D., & Cain, M. S. (2018) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2018.01.017 · Cited by 33
Tsikopoulos, K., Mavridis, D., Georgiannos, D., & Vasiliadis, H. S. (2018) · Clinical Orthopaedics and Related Research · doi:10.1097/01.blo.0000534691.24149.a2 · Cited by 35
Tsikopoulos, K., Sidiropoulos, K., Kitridis, D., Cain, S. M., Metaxiotis, D., & Ali, A. A. (2019) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000000946 · Cited by 34
Koechlin, H., Whalley, B., Welton, N. J., & Locher, C. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1174-6 · Cited by 33
Progress and challenges of network meta‐analysis
Tian, J., Gao, Y., Zhang, J., Yang, Z., Dong, S., Zhang, T., Sun, F., Wu, S., Wu, J., Wang, J., et al. (2021) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12443 · Cited by 29
Tang, Y., Wang, L., He, J., Xu, Y., Huang, S., & Fang, Y. (2021) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s332967 · Cited by 35
Risk-reducing medications for primary breast cancer: a network meta-analysis
Mocellin, S., Goodwin, A., & Pasquali, S. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012191.pub2 · Cited by 40
Liu, X., Qiu, Y., Yu, E. D., Xiang, S., Meng, R., Niu, K. F., & Zhu, H. (2020) · American Journal of Reproductive Immunology · doi:10.1111/aji.13219 · Cited by 29
The effects of exercise on insomnia disorders: An umbrella review and network meta-analysis
Tian, C., Wei, Y., Xu, M., Liu, J., Tong, B., Ning, J., Wang, Y., Wang, Y., Estill, J. A. M., & Ge, L. (2024) · Sleep Medicine · doi:10.1016/j.sleep.2024.02.002 · Cited by 36
Garcia, S. P., Cureau, F. V., Iorra, F. D. Q., Bottino, L. G., Monteiro, L. E. R. C., Leivas, G., Umpierre, D., & Schaan, B. D. (2025) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2025.112027 · Cited by 47
Novel oral anticoagulants for the secondary prevention of cerebral ischemia: a network meta-analysis
Katsanos, A. H., Mavridis, D., Parissis, J., Deftereos, S. G., Frogoudaki, A., Vrettou, A., Ikonomidis, I., Chondrogianni, M. E., Safouris, A., Filippatou, A. G., et al. (2016) · Therapeutic Advances in Neurological Disorders · doi:10.1177/1756285616659411 · Cited by 31
Schwingshackl, L., Chaimani, A., HOFFMANN, G. E., Schwedhelm, C., & Boeing, H. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0455-1 · Cited by 34
“, disagreement between the different sources of evidence) in the data, we will employ both local and global approaches [42].”
MethodSelf-citationHsieh, M., Tseng, P., Wu, Y., Tu, Y., Wu, H., Hsu, C., Lei, W., Stubbs, B., Carvalho, A. F., Liang, C., et al. (2019) · Obesity Reviews · doi:10.1111/obr.12835 · Cited by 33
Efficacy and Safety of First- and Second-Line Drugs to Prevent Glucocorticoid-Induced Fractures
Ding, L., Hu, J., Wang, D., Liu, Q., Mo, Y., Tan, X., & Wen, F. (2019) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgz023 · Cited by 32
Deng, J., Silver, Z. A., Huang, E., Zheng, E., Kavanagh, K., Wen, A., Cheng, W., Dobransky, J. S., Sanger, S., & Grammatopoulos, G. A. (2020) · Rheumatology · doi:10.1093/rheumatology/keaa228 · Cited by 30
Tokavanich, N., Prasitlumkum, N., Mongkonsritragoon, W., Cheungpasitporn, W., Thongprayoon, C., Vallabhajosyula, S., & Chokesuwattanaskul, R. (2021) · Scientific Reports · doi:10.1038/s41598-021-91610-8 · Cited by 30
Kishi, T., Ikuta, T., Sakuma, K., Hatano, M., Matsuda, Y., Kito, S., & Iwata, N. (2023) · Molecular Psychiatry · doi:10.1038/s41380-023-02045-8 · Cited by 30
Ballesio, A., Zagaria, A., Vacca, M., Pariante, C. M., & Lombardo, C. (2023) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2023.05.006 · Cited by 32
“Additionally, although for each observed pairwise comparison the presence of publication bias was not detected through the standard approach (e. g., Salanti et al., 2014), it should be noted that we were unable to run funnel plots for publication bias detection in several pairwise meta -an- alyses…”
Gibbs, V. N., Champaneria, R., Sandercock, J., Welton, N. J., Geneen, L. J., Brunskill, S. J., Doree, C. J., Kimber, C., Palmer, A. J. R., & Estcourt, L. J. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013295.pub2 · Cited by 39
Glue, P., & Herbison, P. (2015) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867415614109 · Cited by 31
“…criticism, that patients in combination studies might have different placebo-associated responses which might affect the assumption of transitivity (Salanti et al., 2014), is difficult to address with this study-level dataset; however, the lack of network inconsistency in the these analyses…”
BackgroundResultTomlinson, A., Efthimiou, O., Boaden, K., New, E., Mather, S., Salanti, G., Imai, H., Ogawa, Y., Tajika, A., Kishimoto, S., et al. (2019) · Evidence-Based Mental Health · doi:10.1136/ebmental-2019-300087 · Cited by 29
Self-citationZhang, L., Wang, F., Tashiro, S., & Liu, P. J. (2024) · Advances in Nutrition · doi:10.1016/j.advnut.2024.100330 · Cited by 42
Metadata
- Authors
- Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T.
- Affiliations
- University of Ioannina (GR)
- University of Bristol (GB)
- Corresponding author
- Higgins, Julian P. T.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-07-03
- Volume
- 9
- Issue
- 7
- Pages
- e99682
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4084629
- OpenAlex
- W2074147400
- Cited by
- 1,402
- References
- 32
- Citations by year
- 2026: 116 · 2025: 181 · 2024: 150 · 2023: 144 · 2022: 151 · 2021: 170 · 2020: 152 · 2019: 115 · 2018: 96 · 2017: 56 · 2016: 46 · 2015: 17 · 2014: 8
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/K025643/1, MR/K023233/1, G0902118)
- FP7 Ideas: European Research Council (260559)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, indirect comparison, transitivity assumption, direct and indirect evidence, treatment ranking, GRADE framework, topical antibiotics, chronic otitis media, tympanic membrane perforation
- MeSH terms
- Humans, Models, Theoretical, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.