The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
Open accessAbstract
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [45].”
From The Effect of Different Psychological Interventions on Athletes' Competitive Anxiety: A Systematic Review and Bayesian Network Meta-Analysis. (2026) · cites this paper as [45]
In the citing paper
“Therefore, we downgraded the quality of evidence for the comparison between SGLT2i and finerenone, SGLT2i and GLP-1 RA.”
In the citing paper
“Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head.”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
In the citing paper
“Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Items 501–600 of 1,001
Gudivada, K. K., Kumar, A. R., Sriram, K., Baby, J., Shariff, M., Sampath, S., Sivakoti, S., & Krishna, B. (2021) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2021.12.015 · Cited by 27
Li, K., Wang, D., Zhang, X., Yang, J., & Chen, X. (2022) · Nutrition & Dietetics · doi:10.1111/1747-0080.12721 · Cited by 27
Interventions for hereditary haemochromatosis
Buzzetti, E., Kalafateli, M., Thorburn, D., Davidson, B. R., Tsochatzis, E. A., & Gurusamy, K. S. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011647.pub2 · Cited by 24
Pharmacological interventions for acute hepatitis B infection
Mantzoukis, K., Rodríguez‐Perálvarez, M., Buzzetti, E., Thorburn, D., Davidson, B. R., Tsochatzis, E. A., & Gurusamy, K. S. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011645.pub2 · Cited by 27
Meta-Review: Network Meta-Analyses in Child and Adolescent Psychiatry
Cortese, S., Tomlinson, A., & Cipriani, A. (2018) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2018.07.891 · Cited by 25
Facciorusso, A., Papagiouvanni, I., Cela, M., Buccino, V. R., & Sacco, R. (2019) · Liver International · doi:10.1111/liv.14109 · Cited by 25
Schwarz, E. I., Scherff, F., Haile, S. R., Steier, J. S., & Kohler, M. (2019) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.8092 · Cited by 25
Zhou, H., Zheng, S., Li, A., Gao, Q., Ou, Q., Chen, Y., Wu, S., Lin, D., Liu, S., Huang, L., et al. (2020) · EClinicalMedicine · doi:10.1016/j.eclinm.2020.100422 · Cited by 25
Srisurapanont, M., Likhitsathian, S., Suttajit, S., Maneeton, N., Maneeton, B., Oon‐arom, A., & Suradom, C. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2020.108467 · Cited by 26
Tian, H., Chen, Y., Sun, M., Huang, L., Xu, G., Yang, C., Luo, Q., Zhao, L., Wei, Z., & Liang, F. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1071326 · Cited by 26
“According to the GRADE tool, the quality of the two outcomes (CFS and PSQI) was moderately to critically low.”
BackgroundInfluentialSoe, K. K., Krikeerati, T., Pheerapanyawaranun, C., Niyomnaitham, S., Phinyo, P., & Thongngarm, T. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1184552 · Cited by 25
“Two reviewers (KS and TK) independently graded the certainty evidence for each outcome using the Grading of Recommended Assessment, Development, and Evaluation (GRADE) approach (Puhan et al., 2014).”
MethodA pulmonologist's guide to understanding network meta-analysis
Dobler, C. C., Wilson, M. E., & Murad, M. H. (2018) · European Respiratory Journal · doi:10.1183/13993003.00525-2018 · Cited by 24
Self-citationRosenberger, K. J., Xing, A., Murad, M. H., Chu, H., & Lin, L. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06357-1 · Cited by 24
“The non-informative prior N(0, 1002) was used for all treatment effects (log ORs), and we assumed consistency between direct and indirect evidence in each NMA.”
BackgroundInfluentialSelf-citationWeibel, S., Pace, N. L., Schaefer, M. S., Raj, D., Schlesinger, T., Meybohm, P., Kienbaum, P., Eberhart, L. H. J., & Kranke, P. (2021) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12429 · Cited by 24
“The 585 included RCTs comprising 97 516 randomized participants were mostly of small size with a median (IQR [range]) number of 100 (70-160 [20-5199]) participants, published between 1965 and 2017 (with 71% from 2000 onward), and primarily conducted in Asia (51%), Europe (25%), andNorthAmerica (16%).”
BackgroundRajadurai, S. G., Maharajan, M. K., Veettil, S. K., & Gopinath, D. (2021) · Journal of Fungi · doi:10.3390/jof7080637 · Cited by 24
“In total, 14 randomized control trials [13–16,20,40–43,45,49,50,52,53] with 2760 participants comparing 8 interventions (Figure 3) were included in this NMA, which is expressed as the risk ratio (RR) of achieving clinical cure when treated with the specified antifungal agent as compared to fluconazole.”
MethodInfluentialYang, M., He, Q., Gao, F., Nirantharakumar, K., Veenith, T., Qin, X. S., Page, A. T., Wong, M. C. S., Huang, J., Kuo, Z. C., et al. (2021) · BMC Medicine · doi:10.1186/s12916-021-02180-5 · Cited by 25
“The estimated effect of the meta-analysis was graded as the moderate quality of evidence based on the GRADE system.”
MethodInfluentialYin, Z., Wang, F., Sun, M., Zhao, L., & Liang, F. (2022) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2022/3178154 · Cited by 25
“+e Grades of Recommendations, Assessment, Development, and Evaluation (GRADE) was used to assess the overall quality of the evidence [35, 36] and ranked it as “high,” “moderate,” “low,” and “critically low.”
MethodEffects of therapies for Ebola virus disease: a systematic review and network meta-analysis
Gao, Y., Zhao, Y., Guyatt, G. H., Fowler, R., Kojan, R., Ge, L., Tian, J., Diaz, J. V., Lado, M., Youkee, D., et al. (2022) · The Lancet Microbe · doi:10.1016/s2666-5247(22)00123-9 · Cited by 23
Self-citationChen, C.-Y., Zhang, E., Zhu, C., Wei, R., Ma, L., Xiu, D., Li, R., Sun, F., Zhou, Y., Cui, Y., et al. (2022) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2022.07.009 · Cited by 26
Network Meta-analysis of Different Treatments for Vestibular Migraine
Chen, J.-J., Zeng, B., Su, K., Wu, Y., Tu, Y., Stubbs, B., Chen, T., Zeng, B., Chen, Y., Hsu, C., et al. (2023) · CNS Drugs · doi:10.1007/s40263-023-01037-0 · Cited by 27
Kim, M. S., Kim, J. Y., Song, Y. S., Hong, S., Won, H., Kim, W. J., Kwon, Y., Ha, J., Fiedorowicz, J. G., Solmi, M., et al. (2023) · Obesity Reviews · doi:10.1111/obr.13670 · Cited by 23
“In accordance with previous umbrella reviews, 33 – 36 we assessed the certainty of evidence for the 82 associations using the GRADE framework 37 based on the results of our re-analyses.”
MethodFILIPPO, O. D., Piroli, F., Bruno, F., Bocchino, P. P., Saglietto, A., Franchin, L., Angelini, F., Gallone, G., Rizzello, G., Ahmad, M., et al. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112476 · Cited by 26
Prat, L. I., Wilson, P., Freeman, S. C., Sutton, A. J., Cooper, N. J., Roccarina, D., Benmassaoud, A., Torres, M. C. P., Hawkins, N., Cowlin, M., et al. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013120.pub2 · Cited by 27
Veettil, S. K., Jinatongthai, P., Nathisuwan, S., Teerawattanapong, N., Ching, S. M., Lim, K. G., Saokaew, S., Phisalprapa, P., Reid, C. M., & Chaiyakunapruk, N. (2018) · Clinical Epidemiology · doi:10.2147/clep.s174120 · Cited by 24
“The quality of evidence from NMA was evaluated using GRADEpro GDT software online.(45) Description of grading of evidence is provided in Methods S2.”
MethodMohan, B. P., Khan, S. R., Kassab, L. L., Ponnada, S., Dulai, P. S., & Kochhar, G. S. (2020) · Endoscopy International Open · doi:10.1055/a-1236-3007 · Cited by 24
“The quality of evidence was rated for results from the meta-analysis according to the GRADE working group approach [27].”
Zhang, S., Xu, T., Shi, Q., Li, S., Wang, L., An, Z., & Su, N. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.698437 · Cited by 28
Furukawa, T. A., Miura, T., Chaimani, A., Leucht, S., Cipriani, A., Noma, H., Mitsuyasu, H., Kanba, S., & Salanti, G. (2016) · BMC Research Notes · doi:10.1186/s13104-016-2019-1 · Cited by 23
“[2] rates the quality of evidence separately for direct and indirect estimates, and each rating is more impressionistic than quantitative.”
BackgroundMethodLaohapiboolrattana, W., Jansem, P., Anukoolwittaya, P., Roongpiboonsopit, D., Hiransuthikul, A., Pongpitakmetha, T., Thanprasertsuk, S., & Rattanawong, W. (2024) · The Journal of Headache and Pain · doi:10.1186/s10194-024-01904-1 · Cited by 23
“Judg-ments across these domains were summarized to obtain four levels of confidence for each relative treatment effect, namely very low, low, moderate, or high, aligning with the standard Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessment [20].”
MethodVeroniki, A. A., Straus, S. E., Ashoor, H. M., Hamid, J. S., Hemmelgarn, B. R., Holroyd-Leduc, J. M., Majumdar, S. R., McAuley, G., & Tricco, A. C. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010251 · Cited by 22
“This is because it has been suggested that combining IPD with aggregate data minimises the chances of confounding bias in aggregate data NMA.”
MethodInfluentialManagement of frailty: a protocol of a network meta-analysis of randomized controlled trials
Negm, A. M., Kennedy, C. C., Thabane, L., Veroniki, A. A., Adachi, J. D., Richardson, J. A., Cameron, I. D., Giangregorio, A., & Παπαϊωάννου, Α. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0522-7 · Cited by 23
“For instance, if one of the direct comparisons will be rated as low and the other will be rated as moderate evidence, we will rate the quality of indirect evidence as low [45].”
BackgroundMethodInfluentialMen, P., Qu, S., Luo, W., Li, C., & Zhai, S. (2019) · Diabetes Obesity and Metabolism · doi:10.1111/dom.13871 · Cited by 23
“The Grading of Recommendations and Assessment, Development, and Evaluation (GRADE) approach was applied to evaluate the evidence quality of results.(25)”
MethodYogendrakumar, V., Lun, R., Khan, F., Salottolo, K., Lacut, K., Graham, C., Dennis, M. S., Hutton, B., Wells, P. S., Fergusson, D. A., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234957 · Cited by 24
“We judged the quality of evidence for all outcomes using a framework developed by the GRADE working group specifically designed for randomized and non-randomized studies in the context of a network metaanalysis [20].”
MethodYasuda, H., Okano, H., Mayumi, T., Nakane, M., & Shime, N. (2021) · Journal of Intensive Care · doi:10.1186/s40560-021-00539-7 · Cited by 22
“Covariance between two estimates from the same study shows the variance of data in the shared arm, as calculated in a multivariable meta-analysis performed using the GRADE Working Group Approach for an NMA [30, 31].”
MethodChang, P., Chen, K.-H., Jhou, H., Lee, C., Chou, S., Chen, P., & Chang, T. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12040512 · Cited by 24
“The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) criteria were used for the quality of evidence in this network meta-analysis [49,51].”
BackgroundMethodLi, L., Xie, H., Wang, L., Zhang, A., Mou, X., Lin, Y., Ma, H., Wang, Y., Li, J., Gao, J., et al. (2022) · Chinese Medicine · doi:10.1186/s13020-022-00600-z · Cited by 25
Pennisi, G., Celsa, C., Enea, M., Vaccaro, M., Marco, V. D., Ciccioli, C., Infantino, G., Mantia, C. L., Parisi, S., Vernuccio, F., et al. (2022) · Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2022.07.001 · Cited by 23
Hussain, N., Brull, R., Vannabouathong, C., Speer, J., Lagnese, C. M., McCartney, C. J. L., & Abdallah, F. W. (2022) · Anaesthesia · doi:10.1111/anae.15873 · Cited by 23
“Evidence was further downgraded if substantial heterogeneity was present in direct and indirect estimates per GRADE criteria [11].”
BackgroundGiri, S., Jearth, V., Seth, V., Darak, H., & Sundaram, S. (2023) · Clinical and Experimental Hepatology · doi:10.5114/ceh.2023.126077 · Cited by 24
“The assessment of the certainty of the evidence for all evaluable outcomes was done using the Confidence in Network Meta-Analysis (CINeMA) web application and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for network meta-analysis [16, 17].”
Ramaswamy, V. V., Bandyopadhyay, T., Abiramalatha, T., Pullattayil, A. K., Szczapa, T., Wright, C. J., & Roehr, C. C. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102097 · Cited by 23
Sud, S., Fan, E., Adhikari, N. K. J., Friedrich, J. O., Ferguson, N. D., Combes, A., Guérin, C., & Guyatt, G. H. (2024) · Intensive Care Medicine · doi:10.1007/s00134-024-07492-7 · Cited by 24
“To assess the certainty of evidence for direct comparisons we used standard GRADE methodology [46–48] and language [49] adapted for network meta-analysis [50, 51].”
Self-citationInduction immunosuppression in adults undergoing liver transplantation: a network meta-analysis
Best, L. M., Leung, J., Freeman, S. C., Sutton, A. J., Cooper, N. J., Milne, E., Cowlin, M., Payne, A., Walshaw, D., Thorburn, D., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013203.pub2 · Cited by 24
“[DOI: 10.1002/14651858.CD010253.pub2] Puhan 2014 Puhan MA, Schünemann HJ, Murad MH, Li T, BrignardelloPetersen R, Singh JA, et al. A GRADE Working Group approach for rating the quality of treatment eBect estimates from network meta-analysis.”
MethodInfluentialResearch Methods in Learning Design and Technology
Romero‐Hall, E. (2020) · Routledge eBooks · doi:10.4324/9780429260919 · Cited by 25
Ha, V., Bonner, A., Jadoo, J. K., Beyene, J., Anand, S. S., & de Souza, R. J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0182095 · Cited by 25
Efficacy of topical agents in oral mucositis prevention: Systematic review and network meta‐analysis
Coppini, M., Caponio, V. C. A., Mauceri, R., Bizzoca, M. E., Laino, L., Lorenzo‐Pouso, A. I., Russo, D., Troiano, G., Silva, F. F. V. E., Muzio, L. L., et al. (2024) · Oral Diseases · doi:10.1111/odi.15046 · Cited by 24
Pharmacologic Management of Acute Pain in Children
Olejnik, L. J., Lima, J. P., Sadeghirad, B., Busse, J. W., Flórez, I. D., Ali, S., Bunker, J., Jomaa, D., Bleik, A., & Eltorki, M. (2025) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2024.5920 · Cited by 23
Menzies, R., Li, A. L. K., Melamed, N., Shah, P. S., Horn, D. D., Barrett, J. F. R., & Murphy, K. E. (2020) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2020.02.003 · Cited by 22
Erre, G. L., Sebastiani, M., Fenu, M. A., Zinellu, A., Floris, A., Cavagna, L., Renzoni, E. A., Manfredi, A., Passiu, G., Woodman, R. J., et al. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9082560 · Cited by 22
“We downgraded by 1 point in the presence of serious concerns and by 2 points in the presence of very serious concerns [17].”
BackgroundLai, C.-J., Yeh, Y., Tu, Y., Cheng, Y., Liu, C., & Fan, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-94114-7 · Cited by 22
Jongkhajornpong, P., Anothaisintawee, T., Lekhanont, K., Numthavaj, P., McKay, G. J., Attia, J., & Thakkinstian, A. (2021) · Cornea · doi:10.1097/ico.0000000000002943 · Cited by 23
Gong, J.-Y., Ren, H., Peng, S.-Y., Xing, K., Li, F., Liu, M.-Z., Luo, Z., & Luo, J. (2022) · Phytomedicine · doi:10.1016/j.phymed.2022.153942 · Cited by 22
Mitra, S., Gardner, C. E., MacLellan, A. D., Disher, T. C., Styranko, D. M., Campbell-Yeo, M. L., Kuhle, S., Johnston, B. C., & Dorling, J. S. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013846.pub2 · Cited by 21
Yang, B., Tseng, P., Hu, X., Zeng, B., Chang, J. P., Liu, Y., Chu, W.-J., Zhang, S., Zhou, Z.-L., Chu, C., et al. (2022) · Progress in Lipid Research · doi:10.1016/j.plipres.2022.101196 · Cited by 23
Tang, S., Ruan, Z., Ma, A., Wang, D., & Kou, J. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.1063349 · Cited by 22
Isaacs, A. M., Yang, R., Cadieux, M., Ben‐Israel, D., Sader, N., Opoku-Darko, M., Frizon, L. A., Yong, H. Y. F., Premji, Z., Nagel, S. J., et al. (2023) · Neurosurgical FOCUS · doi:10.3171/2023.1.focus22637 · Cited by 22
Sanchez‐Ramos, L., Lin, L., Vilchez-Lagos, G., Duncan, J. R., Condon, N. A., Wheatley, J., & Kaunitz, A. M. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2022.01.005 · Cited by 22
Single-Shot Regional Anesthesia for Bariatric Surgery: a Systematic Review and Network Meta-Analysis
CASSAI, A. D., Paganini, G., Pettenuzzo, T., Zarantonello, F., Boscolo, A., Tulgar, S., Carron, M., Munari, M., & Navalesi, P. (2023) · Obesity Surgery · doi:10.1007/s11695-023-06737-6 · Cited by 22
“We used the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach to assess the certainty of evidence related to each of the outcomes [10].”
MethodYang, Y., Lang, Z., Wang, X., Yang, P., Meng, N., Yang, X. H., & Liu, Y. (2024) · BMC Anesthesiology · doi:10.1186/s12871-024-02609-3 · Cited by 22
“Nevertheless, this level of evidence may be reduced based on limitations observed in five key aspects: risk of bias, consistency, indirectness, imprecision, and publication bias [10].”
BackgroundTäger, T., Fröhlich, H., Seiz, M., Katus, H. A., & Frankenstein, L. (2019) · Heart Failure Reviews · doi:10.1007/s10741-019-09771-8 · Cited by 22
Efficacy of Treatments for Polycystic Ovarian Syndrome Management in Adolescents
Khalifah, R. A. A., Flórez, I. D., Zoratti, M. J., Dennis, B. B., Thabane, L., & Bassilious, E. (2020) · Journal of the Endocrine Society · doi:10.1210/jendso/bvaa155 · Cited by 23
“Additionally, for network meta-analysis (NMA) the evidence can be rated down for intransitivity, and incoherence [27, 28].”
BackgroundMohan, B. P. (2020) · Annals of Gastroenterology · doi:10.20524/aog.2020.0542 · Cited by 22
Wu, J., Wu, J., Tang, B., Zhang, Z., Wei, F., Yu, D., Li, L., Zhao, Y., Wang, B., Wu, W., et al. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1373691 · Cited by 28
Welch, V. A., Awasthi, S., Cumberbatch, C., Fletcher, R. H., McGowan, J., Merritt, K., Krishnaratne, S., Sohani, S., Tugwell, P., & Wells, G. (2013) · Campbell Systematic Reviews · doi:10.1002/cl2.115 · Cited by 21
Dimou, C., Antza, C., Akrivos, E., Doundoulakis, I., Stabouli, S., Haidich, A., & Kotsis, V. (2018) · Journal of Human Hypertension · doi:10.1038/s41371-018-0138-y · Cited by 21
“NMA was carried out with R version 3.3.3., using package “netmeta.””
BackgroundInfluentialPillastrini, P., Castellini, G., Chiarotto, A., Fasciani, F., Marzioni, F., Vanti, C., Bertozzi, L., & Gianola, S. (2019) · Medicine · doi:10.1097/md.0000000000016762 · Cited by 22
“We will use the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach to evaluate the quality of the body of evidence contributed to the network.([32,33]) We will appraise the GRADE domains (study limitations, indirectness, inconsistency, imprecision and publication bias).”
MethodFang, M., Liu, G., Luo, G., & Wu, T. (2019) · International Wound Journal · doi:10.1111/iwj.13224 · Cited by 21
“controlled trial (RCT) (74 cases),(40) and 4 non-randomised controlled trials (NRCTs) (574 cases),(41-44) describing a total of 20 868 patients (mean age, 36–56 years), derived from China (7), United States (6), Italy (5), Germany (5), Korea (2), Japan (2), Spain (1), Poland (1), France (1), United Arab Emirates (1),…”
BackgroundCho, Y., Choi, G. J., Ahn, E. J., & Kang, H. (2021) · PLoS ONE · doi:10.1371/journal.pone.0243865 · Cited by 22
Effect of rituximab dose on induction therapy in ABO-incompatible living kidney transplantation
Lee, H. R., Kim, K., Lee, S. W., Song, J. H., Lee, J. H., & Hwang, S. D. (2021) · Medicine · doi:10.1097/md.0000000000024853 · Cited by 21
Du, Y., Biao, D., Diao, Y. P., Yin, Z., Li, J., Shu, Y., Zhang, Z., & Chen, L. (2021) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2021.102664 · Cited by 21
Park, S.-K., Lim, T., Cho, H.-Y., Yoon, H., Lee, H., Lee, J., Yoo, S., Kim, J., & Kim, W. H. (2021) · Scientific Reports · doi:10.1038/s41598-021-91314-z · Cited by 23
Lozanovski, V. J., Ramouz, A., Aminizadeh, E., Ali‐Hasan‐Al‐Saegh, S., Khajeh, E., Probst, H., Picardi, S., Rupp, C., Chang, D., Probst, P., et al. (2021) · BJS Open · doi:10.1093/bjsopen/zrab130 · Cited by 20
Wang, Y., Liu, W., Chen, J., Bai, J., Yu, H., Ma, H., Jiang, R., & Xu, G. (2023) · Therapeutic Advances in Chronic Disease · doi:10.1177/20406223231168754 · Cited by 22
Kim, H., Choi, C. U., Rhew, K., Park, J., Lim, Y., Kim, M. G., & Kim, K. (2024) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2024.117490 · Cited by 21
Moura, M. C., Novaes, M. R. C. G., Zago, Y. S. S. P., Eduardo, E. J., & Casulari, L. A. (2016) · Journal of Clinical Medicine Research · doi:10.14740/jocmr2495w · Cited by 21
“The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) model [12] was used to access the quality of the studies.”
MethodFlórez, I. D., Khalifah, R. A. A., Sierra, J. M., Granados, C., Yepes-Núñez, J. J., Cuello‐García, C. A., Pérez‐Gaxiola, G., Zea, A. M., Hernández, G., Veroniki, A. A., et al. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0186-8 · Cited by 23
“For this purpose, the GRADE approach recommends to assess the incoherence (or inconsistency as described in the “The network metaanalysis: the direct and the indirect evidence” section) criteria, which is defined as the differences between direct and indirect estimates of effect [54].”
InfluentialSelf-citationApplying GRADE to a network meta-analysis of antidepressants led to more conservative conclusions
Bonner, A., Alexander, P. E., Brignardello‐Petersen, R., Furukawa, T. A., Siemieniuk, R. A. C., Zhang, Y., Wiercioch, W., Flórez, I. D., Fei, Y., Agarwal, A., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.05.009 · Cited by 20
Self-citationKim, D. K., Lee, J. Y., Jung, J. H., Hah, Y. S., Koo, K. C., Lee, K. S., Chung, B. H., & Cho, K. S. (2019) · Scientific Reports · doi:10.1038/s41598-019-41412-w · Cited by 21
Li, Y., Du, F., & Fu, D. (2019) · Saudi Journal of Gastroenterology · doi:10.4103/sjg.sjg_538_18 · Cited by 21
Yuan, J.-Q., Yang, M. G., Threapleton, D. E., Qi, X., Ye, D., Mao, C., Tang, J.-L., & Higgins, J. P. T. (2016) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.13776 · Cited by 21
“The strength of evidence was evaluated using the Grading of Recommendations Assessment, Development and Evaluation system (GRADE).(26) The quality of evidence was graded as ‘High’, ‘Moderate’, ‘Low’ and ‘Very low’ based on the fundamental study design and additional methodological factors.”
BackgroundMethodInfluentialNoori, A., Sadeghirad, B., Wang, L., Siemieniuk, R. A. C., Shokoohi, M., Kum, E., Jeddi, M., Montoya, L., Hong, P. J., Zhou, E., et al. (2022) · British Journal of Anaesthesia · doi:10.1016/j.bja.2022.05.031 · Cited by 19
Self-citationSabeti, M. A., Ihsan, M. S., & Aminoshariae, A. (2023) · Journal of Endodontics · doi:10.1016/j.joen.2023.11.007 · Cited by 20
Ōnishi, A., & Furukawa, T. A. (2016) · Springer eBooks · doi:10.1007/978-3-319-25655-9_13 · Cited by 20
Cohen, A. T., Berger, S. E., Milenković, D., Hill, N. R., & Lister, S. (2019) · Pharmacological Research · doi:10.1016/j.phrs.2019.03.017 · Cited by 21
Biofeedback in pediatric populations: a systematic review and meta-analysis of treatment outcomes
Darling, K. E., Benore, E. R., & Webster, E. E. (2019) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz124 · Cited by 22
Miao, Z., Lü, J., Yan, J., Lu, L., Ye, B., & Gu, M. (2020) · Clinical Therapeutics · doi:10.1016/j.clinthera.2020.04.014 · Cited by 21
Subramani, Y., Nagappa, M., Kumar, K., Fochesato, L.-A., Chohan, M. B. Y., Zhu, Y. F., Armstrong, K., & Singh, S. (2020) · BMC Anesthesiology · doi:10.1186/s12871-020-01116-5 · Cited by 22
Jin, L., Guo, S., Zhang, X., Mo, Y., Ke, S., & Duan, C. (2020) · Neurosurgical Review · doi:10.1007/s10143-020-01403-2 · Cited by 20
Lee, H. W., Park, J., Jang, E. J., & Lee, C. (2020) · Respiratory Research · doi:10.1186/s12931-020-01540-8 · Cited by 20
“Until now, it has not been clarified whether there are any differences in pharmacologic effects between individual drugs or their combinations within the same drug class other than LAMA/ LABA in the treatment of COPD patients [43].”
BackgroundResultInfluentialMoroni, F., Baldetti, L., Kabali, C., Briguori, C., Maioli, M., Toso, A., Brilakis, E. S., Gurm, H. S., Bagur, R., & Azzalini, L. (2021) · Journal of the American Heart Association · doi:10.1161/jaha.121.021342 · Cited by 20
“8,9 CLINICAL PERSPECTIVE What Is New?”
Gao, L., Zhao, X., Jiao, L., & Tang, L. (2021) · Eye and Vision · doi:10.1186/s40662-021-00261-3 · Cited by 22
Shiue, Y., Tseng, P., Zeng, B., Chen, Y., Yang, C., Su, K., Chen, T., Wu, Y., Tu, Y., Lin, P., et al. (2022) · Current Neuropharmacology · doi:10.2174/1570159x20666220420122322 · Cited by 21
Sakuraya, M., Okano, H., Yoshihiro, S., Niida, S., & Kimura, K. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.960135 · Cited by 20
Maiti, R., Mishra, A., Srinivasan, A., & Mishra, B. R. (2023) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13568 · Cited by 24
Storytelling in the context of vaccine refusal: a strategy to improve communication and immunisation
Cawkwell, P. B., & Oshinsky, D. M. (2015) · Medical Humanities · doi:10.1136/medhum-2015-010761 · Cited by 22
Veettil, S. K., Teerawattanapong, N., Ching, S. M., Lim, K. G., Saokaew, S., Phisalprapa, P., & Chaiyakunapruk, N. (2017) · OncoTargets and Therapy · doi:10.2147/ott.s127335 · Cited by 22
“The level of evidence supporting the efficacy of these CPAs using GRADE methodology adapted for NMA was high for celecoxib, moderate for low-dose aspirin and low for calcium.(65) COX-2 inhibitors (celecoxib 400–800 mg/day) had a greater protective effect than low-dose aspirin and calcium in reducing the incidence of a…”
MethodOrjuela, N. G., Prieto‐Pinto, L., Lasalvia, P., Herrera, D., Castrillon, J., González-Bravo, D. C., Castañeda‐Cardona, C., & Rosselli, D. (2019) · Dermatologic Therapy · doi:10.1111/dth.13145 · Cited by 20
“ity of evidence for each outcome using the GRADE methodology for NMA (Puhan et al., 2014; Yepes-Nuñez et al., 2019).”
MethodInfluentialPharmacological interventions for primary sclerosing cholangitis
Saffioti, F., Gurusamy, K. S., Hawkins, N., Toon, C. D., Tsochatzis, E. A., Davidson, B. R., & Thorburn, D. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011343.pub2 · Cited by 22
Management and outcomes of obturator hernias: a systematic review and meta-analysis
Burlá, M. M., Gomes, C. P., Calvi, I. P., Oliveira, E. S. C., da Hora, D. A. B., Mao, R.-M. D., de Figueiredo, S. M. P., & Lu, R. (2023) · Hernia · doi:10.1007/s10029-023-02808-w · Cited by 19
Liu, H., Wu, M., Huang, H., Chen, X., Zeng, P., & Xu, Y. (2024) · Ageing Research Reviews · doi:10.1016/j.arr.2024.102508 · Cited by 21
Calabrese, L., Cartocci, A., Rubegni, P., French, L. E., & Kendziora, B. (2025) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.20617 · Cited by 19
Morche, J., Freitag, S., Hoffmann, F., Rissling, O., Langer, G., Nußbaumer-Streit, B., Toews, I., Gartlehner, G., & Meerpohl, J. J. (2020) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2019.11.003 · Cited by 20
Metadata
- Authors
- Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group
- Affiliations
- University of Zurich (CH)
- Mayo Clinic (US)
- Johns Hopkins University (US)
- University of Alabama (US)
- Maastricht University (NL)
- McMaster University (CA)
- Corresponding author
- Puhan, M. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2014-09-24
- Volume
- 349
- Issue
- sep24 5
- Pages
- g5630-g5630
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155798268
- Cited by
- 1,812
- References
- 39
- Citations by year
- 2026: 160 · 2025: 177 · 2024: 216 · 2023: 175 · 2022: 205 · 2021: 276 · 2020: 201 · 2019: 128 · 2018: 96 · 2017: 72 · 2016: 63 · 2015: 36 · 2014: 4
- FWCI
- 57.15
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-06-17)
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, GRADE, treatment effect estimates, comparative effectiveness
- MeSH terms
- Humans, Quality Assurance, Health Care, Meta-Analysis as Topic, Practice Guidelines as Topic, Evidence-Based Medicine, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review