The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
Open accessAbstract
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [45].”
From The Effect of Different Psychological Interventions on Athletes' Competitive Anxiety: A Systematic Review and Bayesian Network Meta-Analysis. (2026) · cites this paper as [45]
In the citing paper
“Therefore, we downgraded the quality of evidence for the comparison between SGLT2i and finerenone, SGLT2i and GLP-1 RA.”
In the citing paper
“Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head.”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
In the citing paper
“Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Items 301–400 of 1,001
Tsai, S., Chung, H., Tseng, P., Wu, Y., Tu, Y., Hsu, C., & Lei, W. (2020) · Medicine · doi:10.1097/md.0000000000019403 · Cited by 46
“We evaluated the quality of evidence with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system.([14])”
MethodGao, Q., Xie, Y.-J., Chen, Y., Tan, H., Guo, Q., & Lau, B. W.-M. (2020) · Neural Regeneration Research · doi:10.4103/1673-5374.300341 · Cited by 51
“According to the GRADE rating standards, which contain five categories—limitations in study design, inconsistency, indirectness, imprecision, and publication bias (Atkins et al., 2004)—the quality of evidence can be graded as high, moderate, low, or very low (Puhan et al., 2014).”
Chen, J.-J., Chen, Y.-W., Zeng, B., Hung, C., Zeng, B.-S., Stubbs, B., Carvalho, A. F., Thompson, T., Roerecke, M., Su, K., et al. (2021) · EClinicalMedicine · doi:10.1016/j.eclinm.2021.101080 · Cited by 48
Lifestyle modifications for nonalcohol-related fatty liver disease: a network meta-analysis
Buzzetti, E., Linden, A., Best, L. M., Madden, A. M., Roberts, D., Chase, T. J. G., Freeman, S. C., Cooper, N. J., Sutton, A. J., Fritche, D., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013156.pub2 · Cited by 46
Barbato, L., Kalemaj, Z., Buti, J., Baccini, M., Marca, M. L., Duvina, M., & Tonelli, P. (2015) · Journal of Periodontology · doi:10.1902/jop.2015.150363 · Cited by 46
Zhao, X., Xiao, H., Dai, F., Brodie, D., & Meng, L. (2021) · European Respiratory Journal · doi:10.1183/13993003.02928-2020 · Cited by 45
Dhippayom, T., Saensook, T., Promkhatja, N., Teaktong, T., Chaiyakunapruk, N., & Devine, B. (2022) · eClinicalMedicine · doi:10.1016/j.eclinm.2022.101509 · Cited by 45
Li, Y.-K., Xiao, C., Ren, H., Li, W., Guo, Z., & Luo, J. (2025) · European Journal of Pharmacology · doi:10.1016/j.ejphar.2025.177255 · Cited by 47
Ordóñez-Mena, J. M., McCarthy, N. D., & Fanshawe, T. R. (2017) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkx430 · Cited by 47
Mohan, B. P., Khan, S. R., Chandan, S., Kassab, L. L., Ponnada, S., Asokkumar, R., Shen, B., Iacucci, M., & Navaneethan, U. (2020) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2020.06.048 · Cited by 45
Evaluation of Long-term Clearance Rates of Interventions for Actinic Keratosis
Steeb, T., Wessely, A., Petzold, A., Brinker, T. J., Schmitz, L., Leiter, U., Garbe, C., Schöffski, O., Berking, C., & Heppt, M. V. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.2779 · Cited by 45
Lewis, S. R., Macey, R., Lewis, J., Stokes, J. R., Gill, J. R., Cook, J. A., Eardley, W., Parker, M. J., & Griffin, X. L. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013405.pub2 · Cited by 46
Efficacy of COVID-19 Treatments: A Bayesian Network Meta-Analysis of Randomized Controlled Trials
Zhang, C., Jin, H., Wen, Y. F., & Yin, G. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.729559 · Cited by 44
Gartlehner, G., Dobrescu, A. I., Chapman, A. L., Toromanova, A., Emprechtinger, R., Persad, E., Affengruber, L., Pieh, C., Klerings, I., & Wagner, G. (2023) · Annals of Internal Medicine · doi:10.7326/m22-1845 · Cited by 48
Sabeti, M. A., Huang, Y., Chung, Y. J., & Azarpazhooh, A. (2021) · Journal of Endodontics · doi:10.1016/j.joen.2021.08.008 · Cited by 43
Lü, Y., Zhu, H., Wang, Q., Tian, C., Lai, H., Hou, L., Liu, Y., Gao, Y., Liu, M., Yang, F., et al. (2022) · Sleep Medicine · doi:10.1016/j.sleep.2022.03.012 · Cited by 42
CASSAI, A. D., Sella, N., Geraldini, F., Tulgar, S., Ahişkalıoğlu, A., Dost, B., Manfrin, S., Karapınar, Y. E., Paganini, G., Beldağlı, M., et al. (2022) · Korean Journal of Anesthesiology · doi:10.4097/kja.22366 · Cited by 43
“We used the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach to assess the certainty of evidence related to each of the outcomes [16].”
MethodPan, B., Ge, L., Lai, H., Hou, L., Tian, C., Wang, Q., Yang, K., Lü, Y., Zhu, H., Li, M., et al. (2023) · Drugs · doi:10.1007/s40265-023-01859-8 · Cited by 44
“9 Assessing Certainty of Evidence We used the GRADE approach to assess the certainty (quality) of evidence for direct, indirect, and network comparisons using a non-contextualized approach [32, 33].”
Methodvan den Houten, M. M. L., Hageman, D., Gommans, L. N. M., Kleijnen, J., Scheltinga, M. R., & Teijink, J. A. W. (2019) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2018.12.023 · Cited by 42
Ha, J., Kwon, Y., Kwon, J., Kim, D., Park, S., Hwang, J., Lee, C. M., & Park, S. S. (2021) · Obesity Reviews · doi:10.1111/obr.13249 · Cited by 47
D’Antonio, F., Eltaweel, N. A., Prasad, S., Flacco, M. E., Manzoli, L., & Khalil, A. A. (2023) · PLoS Medicine · doi:10.1371/journal.pmed.1004266 · Cited by 45
Antivirals for post-exposure prophylaxis of influenza: a systematic review and network meta-analysis
Zhao, Y., Gao, Y., Guyatt, G. H., Uyeki, T. M., Liu, P., Liu, M., Shen, Y., Chen, X., Luo, S., Li, X., et al. (2024) · The Lancet · doi:10.1016/s0140-6736(24)01357-6 · Cited by 46
Self-citationSandhu, S. S., Cheema, M. S., & Khehra, H. S. (2016) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2015.12.025 · Cited by 43
“73 (11-66) Aspirin preemptive postoperative 6 (2-14) 78.”
InfluentialNetwork meta-analysis: users’ guide for pediatricians
Khalifah, R. A. A., Flórez, I. D., Guyatt, G. H., & Thabane, L. (2018) · BMC Pediatrics · doi:10.1186/s12887-018-1132-9 · Cited by 40
“Although, the scope of the research question can vary from narrow to broad, it is essential that for any paired comparison within the NMA, it is plausible that we will, for each outcome of interest, observe similar effects across all patient populations being addressed [30, 31].”
BackgroundMethodInfluentialSelf-citationSoni, P., & Chawla, E. (2021) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2021.106893 · Cited by 41
Bae, S. J., Alboog, A. S., Esquivel, K. S., Abbasi, A., Zhou, J. J., & Chui, J. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.08.034 · Cited by 43
Tian, Z., Barbosa, C. V., Lang, H., Bauersachs, J. K., Melk, A., & Schmidt, B. M. W. (2023) · Cardiovascular Research · doi:10.1093/cvr/cvad165 · Cited by 44
Pitre, T., Liu, W., Zeraatkar, D., Casey, J. D., Dionne, J. C., Gibbs, K. W., Ginde, A. A., Needham-Nethercott, N., Rice, T. W., Semler, M. W., et al. (2025) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(25)00029-3 · Cited by 44
Pathak, M., Dwivedi, S. N., Deo, S. V. S., Thakur, B., Sreenivas, V., & Rath, G. K. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0754-1 · Cited by 41
“Further, it will also review the quality of evidence using GRADE approach [32].”
BackgroundMethodInfluentialRattanavipanon, W., Nithiphongwarakul, C., Sirisuwansith, P., Chaiyasothi, T., Thakkinstian, A., Nathisuwan, S., & Pathomwichaiwat, T. (2021) · Phytomedicine · doi:10.1016/j.phymed.2021.153512 · Cited by 43
Aldin, A., Besiroglu, B., Adams, A., Monsef, I., Piechotta, V., Tomlinson, E., Hornbach, C., Dressen, N., Goldkuhle, M., Maisch, P., et al. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013798.pub2 · Cited by 40
Yan, H., Shi, J., Li, X., Dai, Y., Wu, Y., Zhang, J., Gu, Z., Zhang, C., & Leng, J. (2022) · Fertility and Sterility · doi:10.1016/j.fertnstert.2022.08.856 · Cited by 42
Gu, Y., Sun, L., Zhang, W., Kong, T., Zhou, R., He, Y., Deng, C., Yang, L., Kong, J., Chen, Y., et al. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1102792 · Cited by 46
“Including specific guidance for network meta-analyses (Puhan et al., 2014).”
BackgroundInfluentialLow, C. J. W., Ling, R. R., Lau, M. P. X. L., Liu, N. S. H., Tan, M., Tan, C. S., Lim, S. L., Rochwerg, B. N., Combes, A., Brodie, D., et al. (2024) · Intensive Care Medicine · doi:10.1007/s00134-023-07278-3 · Cited by 42
“…trials and propensity score-matched studies, and therefore plays a key role to inform future randomized trials on the topic. meta-analyses [46–51], ranking certainty of evidence from “high” (there is confidence that the true effect lies close to the estimate of the effect) to “very low”…”
Kim, D. K., Kim, J. H., Lee, J. Y., Ku, N. S., Lee, H. S., Park, J.-Y., Kim, J. W., Kim, K. J., & Cho, K. S. (2020) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(20)30121-3 · Cited by 40
Phinyo, P., Koompawichit, P., Nochaiwong, S., Tovanabutra, N., Chiewchanvit, S., & Chuamanochan, M. (2020) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2020.08.055 · Cited by 41
Impact of the diagnostic test Xpert MTB/RIF on patient outcomes for tuberculosis
Haraka, F., Kakolwa, M. A., Schumacher, S. G., Nathavitharana, R. R., Denkinger, C. M., Gagneux, S., Reither, K., & Ross, A. J. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012972.pub2 · Cited by 40
The 15-Months Clinical Experience of SARS-CoV-2: A Literature Review of Therapies and Adjuvants
Inchingolo, A. D., Dipalma, G., Inchingolo, A. M., Malcangi, G., Santacroce, L., D’Oria, M. T., Isacco, C. G., Bordea, I. R., Candrea, S., Scarano, A., et al. (2021) · Antioxidants · doi:10.3390/antiox10060881 · Cited by 54
Noninvasive respiratory support after extubation: a systematic review and network meta-analysis
Boscolo, A., Pettenuzzo, T., Sella, N., Zatta, M., Salvagno, M., Tassone, M., Pretto, C., Peralta, A., Muraro, L., Zarantonello, F., et al. (2023) · European Respiratory Review · doi:10.1183/16000617.0196-2022 · Cited by 39
“The GRADE (Grading of Recommendations Assessment, Development and Evaluations) approach was used to assess the certainty of evidence related to the outcomes [35].”
Management of de Quervain Tenosynovitis
Challoumas, D., Ramasubbu, R., Rooney, E., Seymour-Jackson, E., Putti, A., & Millar, N. L. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.37001 · Cited by 39
Sabeti, M. A., Karimpourtalebi, N., Shahravan, A., & Omid Dianat, A. (2024) · Journal of Endodontics · doi:10.1016/j.joen.2024.03.007 · Cited by 39
Conservative Treatments in the Management of Acute Painful Vertebral Compression Fractures
Alimy, A., Anastasilakis, A. D., CAREY, J. J., D’Oronzo, S., Naciu, A. M., Paccou, J., Yavropoulou, M. P., Lems, W. F., & Rolvien, T. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.32041 · Cited by 45
Sekercioglu, N., Veroniki, A. A., Thabane, L., Busse, J. W., Akhtar‐Danesh, N., Iorio, A., Lopes, L. C., & Guyatt, G. H. (2017) · PLoS ONE · doi:10.1371/journal.pone.0171028 · Cited by 39
Self-citationTonin, F. S., Borba, H. H. L., Leonart, L. P., Mendes, A. M., Steimbach, L. M., Pontarolo, R., & Fernández-Llimós, F. (2018) · International Journal of Epidemiology · doi:10.1093/ije/dyy197 · Cited by 40
Al-Rudayni, A. H. M., Gopinath, D., Maharajan, M. K., Veettil, S. K., & Menon, R. K. (2021) · Current Oncology · doi:10.3390/curroncol28040250 · Cited by 47
“The quality of evidence was rated using the Grading of Recommendation, Assessment, Development and Evaluation (GRADE) approach [22].”
MethodNochaiwong, S., Chuamanochan, M., Ruengorn, C., Awiphan, R., Tovanabutra, N., & Chiewchanvit, S. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3237 · Cited by 40
Lewis, S. R., Macey, R., Stokes, J., Cook, J. A., Eardley, W. G., & Griffin, X. L. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013404.pub2 · Cited by 42
Geraldini, F., CASSAI, A. D., Diana, P., Correale, C., Boscolo, A., Zampirollo, S., Disarò, L., Carere, A., Cacco, N., Navalesi, P., et al. (2022) · Neurocritical Care · doi:10.1007/s12028-022-01493-4 · Cited by 39
“We used the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach to assess the certainty of evidence related to each of the outcomes [10].”
MethodKazemi, A., Shim, S. R., Jamali, N., Hassanzadeh-Rostami, Z., Soltani, S., Sasani, N., Mohsenpour, M. A., Firoozi, D., Basirat, R., Hosseini, R., et al. (2022) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2022.110037 · Cited by 38
A clinician’s guide to network meta-analysis
Phillips, M., STEEL, D. H., Wykoff, C. C., Busse, J. W., Bannuru, R. R., Thabane, L., Bhandari, M., Chaudhary, V., for the Retina Evidence Trials InterNational Alliance (R.E.T.I.N.A.) Study Group, Sivaprasad, S., et al. (2022) · Eye · doi:10.1038/s41433-022-01943-5 · Cited by 40
“It is important to note that the evidence provided by an NMA is subject to the limitations of the individual RCTs included within the network [11].”
BackgroundSelf-citationApplication of network meta-analysis in the field of physical activity and health promotion
Su, X., McDonough, D. J., Chu, H., Quan, M., & Gao, Z. (2020) · Journal of Sport and Health Science · doi:10.1016/j.jshs.2020.07.011 · Cited by 38
“To evaluate the certainty of the evidence from NMAs, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) was described by Puhan et al.24 in 2014.”
MethodNeedling Point Location Used in Sham Acupuncture for Chronic Nonspecific Low Back Pain
Lee, B., Kwon, C., Lee, H. W., Nielsen, A., Wieland, L. S., Kim, T., Birch, S., Alræk, T., & Lee, M. S. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.32452 · Cited by 41
Deng, M., Wen, Y., Yan, J., Fan, Y., Wang, Z., Zhang, R., Ren, L., Ba, Y., Wang, H., Lü, Q., et al. (2023) · BMC Medicine · doi:10.1186/s12916-023-03129-6 · Cited by 40
“…comparative out-comes’ certainty in effect estimates using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group recommended method to avoid inflated recommendations for interventions that perform well statistically but have a low level of evidence [30–33].”
MethodHayashi, M., Yamamoto, N., Kuroda, N., Kano, K., Miura, T., Kamimura, Y., & Shiroshita, A. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2024.01.024 · Cited by 44
Brignardello‐Petersen, R., & Guyatt, G. H. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae260 · Cited by 41
Self-citationHuang, I., Wu, P., Yang, T., Li, H., Huang, Y., Cheng, Y., Kuo, P., Lee, Y.-H., Huang, Y.-C., & Tu, Y. (2021) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2021.01.024 · Cited by 38
Wang, J., Zhao, G., Song, G., & Liu, J. (2021) · Journal of Pain Research · doi:10.2147/jpr.s313972 · Cited by 38
Best, L. M., Freeman, S. C., Sutton, A. J., Hawkins, N., Tsochatzis, E. A., & Gurusamy, K. S. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013103 · Cited by 40
Mappa, I., Tartaglia, S., Maqina, P., Makatsariya, A. D., Ghi, T., Rizzo, G., & D’Antonio, F. (2021) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/aogs.14236 · Cited by 40
Khera, R., Schuemie, M. J., Lu, Y., Ostropolets, A., Chen, R., Hripcsak, G., Ryan, P. B., Krumholz, H. M., & Suchard, M. A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-057977 · Cited by 37
Schwingshackl, L., Schwarzer, G., Rücker, G., & Meerpohl, J. J. (2019) · Advances in Nutrition · doi:10.1093/advances/nmz036 · Cited by 37
Piechotta, V., Jakob, T., Langer, P. H., Monsef, I., Scheid, C., Estcourt, L. J., Ocheni, S., Theurich, S., Kuhr, K., Scheckel, B., et al. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013487 · Cited by 35
Yang, X. H., Fang, T., Mobarak, S. A., Wang, J., Wang, C., Huang, S., Jiang, L., Chen, X., Li, H., Chang, W., et al. (2020) · Sexually Transmitted Infections · doi:10.1136/sextrans-2019-054349 · Cited by 37
Facciorusso, A., Singh, S., Fehmi, S. M. A., Annese, V., Lipham, J. C., & Yadlapati, R. H. (2020) · Surgical Endoscopy · doi:10.1007/s00464-020-07920-x · Cited by 37
“We used Grading of Recommendations Assessment, Development and Evaluation (GRADE) criteria for network meta-analysis to appraise quality of evidence [13].”
MethodAlhindi, Y., & Avery, A. (2022) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2022.100944 · Cited by 41
Rapid reviews methods series: Guidance on assessing the certainty of evidence
Gartlehner, G., Nußbaumer-Streit, B., Devane, D., Kahwati, L. C., Viswanathan, M., King, V. J., Qaseem, A., Akl, E. A., & Schuenemann, H. J. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112111 · Cited by 37
Kwak, D., Ha, J., Won, Y., Kwon, Y., & Park, S. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-043007 · Cited by 34
Chaw, S. H., Lo, Y. L., Goh, S. L., Cheong, C. C., Tan, W. K., Loh, P. S., Wong, L. F., & Shariffuddin, I. I. (2021) · Obesity Surgery · doi:10.1007/s11695-021-05564-x · Cited by 36
Hsu, C., Chou, P., Brunoni, A. R., Hung, K., Tseng, P., Liang, C., Carvalho, A. F., Vieta, E., Tu, Y., Lin, P., et al. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105483 · Cited by 38
Li, X., Hao, X., Liu, J., & Huang, J. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112434 · Cited by 36
“19 To evaluate the confidence in the results of NMA, an approach had been developed and recently refined which was called Confidence In Network Meta ‐ Analysis (CINeMA).”
MethodMiao, Y., Luo, Y., Zhao, Y., Liu, M., Wang, H., & Wu, Y. X. (2024) · Journal of Medical Internet Research · doi:10.2196/58013 · Cited by 39
Mitra, S., Flórez, I. D., Tamayo, M. E., Aune, D., Mbuagbaw, L. C. E., Veroniki, A. A., & Thabane, L. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011271 · Cited by 39
“as the differences between direct and indirect estimates of effect.(56)”
BackgroundYeo, Y. H., Hsu, C., Lee, C., Ho, H. J., Lin, J., Wu, M., Liou, J., & Wu, C. (2018) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.14462 · Cited by 36
Treatment for ascites in adults with decompensated liver cirrhosis: a network meta-analysis
Benmassaoud, A., Freeman, S. C., Roccarina, D., Torres, M. C. P., Sutton, A. J., Cooper, N. J., Prat, L. I., Cowlin, M., Milne, E., Hawkins, N., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013123.pub2 · Cited by 38
Advanced airway management in out of hospital cardiac arrest: A systematic review and meta-analysis
White, L., Melhuish, T., Holyoak, R. S., Ryan, T., Kempton, H., & Vlok, R. A. (2018) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2018.09.045 · Cited by 36
“The quality of evidence and confidence in estimates of effect were assessed using the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) approach [11].”
MethodForoutan, N., Hopkins, R. B., Tarride, J., Flórez, I. D., & Levine, M. (2019) · Clinical and investigative medicine · doi:10.25011/cim.v42i1.32393 · Cited by 37
Gao, Y., Ge, L., Ma, X., Shen, X., Liu, M., & Tian, J. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.022 · Cited by 35
The effect of auricular therapy on blood pressure: A systematic review and meta-analysis
Gao, J., Chen, G., He, H., Liu, C., He, Q., Li, J., & Wang, J. (2019) · European Journal of Cardiovascular Nursing · doi:10.1177/1474515119876778 · Cited by 36
He, Y., Zhong, J., Zhou, W. J., Zeng, S., Li, H., Yang, H., & Shan, N. (2019) · Journal of Minimally Invasive Gynecology · doi:10.1016/j.jmig.2019.03.027 · Cited by 36
Zhou, X., Yao, S., Dong, P., Chen, B., Xu, Z., & Wang, H. X. (2020) · Critical Care · doi:10.1186/s13054-020-03090-3 · Cited by 38
Effects of Major Antihypertensive Drug Classes on Erectile Function: a Network Meta-analysis
Farmakis, I. T., Pyrgidis, N., Doundoulakis, I., Mykoniatis, I., Akrivos, E., & Giannakoulas, G. A. (2021) · Cardiovascular Drugs and Therapy · doi:10.1007/s10557-021-07197-9 · Cited by 36
“Still, in the GRADE assessment, evidence on the matter was rated as low or very low (Table 3).”
MethodInfluentialTäger, T., Frankenstein, L., Atar, D., Agewall, S., Frey, N., Grundtvig, M., Clark, A. L., Cleland, J. G. F., & Fröhlich, H. (2021) · Clinical Research in Cardiology · doi:10.1007/s00392-021-01913-z · Cited by 38
Welch, V. A., Ghogomu, E., Hossain, A., Awasthi, S., Bhutta, Z., Cumberbatch, C., Fletcher, R., McGowan, J., Krishnaratne, S., Kristjansson, E., et al. (2016) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.4073/csr.2016.7 · Cited by 35
Dudás, C., Czumbel, L. M., Kiss, S., Gede, N., Hegyi, P. J., Mártha, K., & Varga, G. (2022) · European Journal of Orthodontics · doi:10.1093/ejo/cjac050 · Cited by 35
“We used the Grading of Recommendations Assessment, Development, and Evaluation approach to assessing the certainty of evidence (21).”
MethodLi, S., Li, Y., Xu, C., Tao, S., Sun, H., Wang, Y., Wang, Y., Li, S., & Ma, X. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02295-2 · Cited by 40
“These assessments categorized evidence quality as high, moderate, low or very low 39–41 .”
Skoetz, N., Haque, M., Weigl, A., Kuhr, K., Monsef, I., Becker, I., & Jordan, K. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012775 · Cited by 35
Xie, D., Liao, Y., Yue, J., Zhang, C., Wang, Y., Deng, C., & Chen, L. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-017883 · Cited by 35
“In our review, we applied the GRADE system to our NMA based on the GRADE working group.(19) The methods of rating the quality of direct comparison are the same for GRADE in traditional meta-analysis.”
MethodHur, M. S., Park, S.-K., Yoon, H., Yoo, S., Lee, H., Kim, W. H., Kim, J., Ku, J. H., & Bahk, J.-H. (2019) · Journal of Anesthesia · doi:10.1007/s00540-018-2597-2 · Cited by 35
Njei, B. M., McCarty, T. R., Muniraj, T., Sharma, P., Jamidar, P. A., Aslanian, H. R., Varadarajulu, S. S., & Navaneethan, U. (2020) · Endoscopy International Open · doi:10.1055/a-1005-6366 · Cited by 38
“Although SUCRA probabilities may not easily substitute for clinical judgement on a patient-by-patient basis, focusing on summary effect estimates and GRADE to rate overall quality of evidence allows for better approximations [27].”
BackgroundMethodZeraatkar, D., Cusano, E., Martinez, J. P. D., Qasim, A., Mangala, S., Kum, E., Bartoszko, J. J., Devji, T., Agoritsas, T., Guyatt, G. H., et al. (2022) · BMJ Medicine · doi:10.1136/bmjmed-2021-000036 · Cited by 37
Self-citationAl-Rudayni, A. H. M., Gopinath, D., Maharajan, M. K., Veettil, S. K., & Menon, R. K. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18147418 · Cited by 34
“The Grading of Recommendation, Assessment, Development and Evaluation (GRADE) approach was used to rate the quality of evidence (high, moderate, low and very low) [22].”
MethodOtieno, P., Agyemang, C. O., Wao, H., Wambiya, E. O. A., Ng’oda, M. A., Mwanga, D. M., Oguta, J. O., Kibe, P. M., & Asiki, G. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073652 · Cited by 37
How To Read a Network Meta-analysis
Niño, A. P., & Brignardello‐Petersen, R. (2023) · European Urology Focus · doi:10.1016/j.euf.2023.10.018 · Cited by 37
Self-citationLee, C. Y. Z., Chakranon, P., & Lee, S. W. H. (2019) · Frontiers in Pharmacology · doi:10.3389/fphar.2019.01221 · Cited by 35
Russo, G. I., Scandura, C., Mauro, M. D., Cacciamani, G. E., Albersen, M., Hatzichristodoulou, G., Fode, M. M., Capogrosso, P., Cimino, S., Marcelissen, T. A. T., et al. (2020) · European Urology Focus · doi:10.1016/j.euf.2020.01.002 · Cited by 36
“The quality of the evidence from direct evidence and NMA was assessed by using risk, which is the recommended approach for assessing risk of bias in studies included in GRADE Working Group reviews [10].”
MethodKwon, C., Lee, B., Cheong, M. J., Kim, T., Jang, B.-H., Chung, S.-Y., & Kim, J. W. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.608896 · Cited by 38
“Quality of Evidence We assessed the quality of evidence regarding the effect estimates derived from NMA for the primary outcome measures using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach (41).”
MethodSakuraya, M., Okano, H., Masuyama, T., Kimata, S., & Hokari, S. (2021) · Critical Care · doi:10.1186/s13054-021-03835-8 · Cited by 32
“We rated down for risk of bias, indirectness, inconsistency, and publication bias but did not rate down for imprecision because this occurred at a later step [26, 27].”
MethodFourth-Generation Total Wrist Arthroplasty: A Systematic Review of Clinical Outcomes
Zijlker, H. J. A., Ritt, M. J. P. F., & Beumer, A. (2021) · Journal of Wrist Surgery · doi:10.1055/s-0041-1735840 · Cited by 34
Canullo, L., Pesce, P., Antonacci, D., Ravidà, A., Galli, M., Khijmatgar, S., Tommasato, G., Sculean, A., & FABBRO, M. D. (2021) · Clinical Oral Investigations · doi:10.1007/s00784-021-04192-0 · Cited by 34
Sim, R., Chong, C. W., Loganadan, N. K., Fong, A. Y. Y., Navaravong, L., Hussein, Z. B., Khunti, K., & Lee, S. W. H. (2021) · Diabetic Medicine · doi:10.1111/dme.14780 · Cited by 35
Metadata
- Authors
- Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group
- Affiliations
- University of Zurich (CH)
- Mayo Clinic (US)
- Johns Hopkins University (US)
- University of Alabama (US)
- Maastricht University (NL)
- McMaster University (CA)
- Corresponding author
- Puhan, M. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2014-09-24
- Volume
- 349
- Issue
- sep24 5
- Pages
- g5630-g5630
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155798268
- Cited by
- 1,812
- References
- 39
- Citations by year
- 2026: 160 · 2025: 177 · 2024: 216 · 2023: 175 · 2022: 205 · 2021: 276 · 2020: 201 · 2019: 128 · 2018: 96 · 2017: 72 · 2016: 63 · 2015: 36 · 2014: 4
- FWCI
- 57.15
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-06-17)
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, GRADE, treatment effect estimates, comparative effectiveness
- MeSH terms
- Humans, Quality Assurance, Health Care, Meta-Analysis as Topic, Practice Guidelines as Topic, Evidence-Based Medicine, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
