The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
Open accessAbstract
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [45].”
From The Effect of Different Psychological Interventions on Athletes' Competitive Anxiety: A Systematic Review and Bayesian Network Meta-Analysis. (2026) · cites this paper as [45]
In the citing paper
“Therefore, we downgraded the quality of evidence for the comparison between SGLT2i and finerenone, SGLT2i and GLP-1 RA.”
In the citing paper
“Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head.”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
In the citing paper
“Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Cited by
Sorted by citations
Items 901–1,000 of 1,001
Hoang, K. D., Chen, J., Huang, T., Kang, Y., & Chen, C. (2024) · PLoS ONE · doi:10.1371/journal.pone.0279784 · Cited by 8
Kim, M. S., Lee, I., Natarajan, P., Do, R., Kwon, Y., Shin, J. I., Solmi, M., Kim, J. Y., Won, H., & Park, S. S. (2024) · Obesity Reviews · doi:10.1111/obr.13823 · Cited by 7
Ou, Y., Lin, D., Ni, X., Feng, C., Rong, J., Gao, X., Yu, Y., Liu, X., Zhang, Z., Wang, X. Q., et al. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1464749 · Cited by 8
Hussain, N., Brull, R., Freedenberg, A. T., Vannabouathong, C., D'Souza, R. S., Havlik, S., Beachy, P., Gunawan, A., Weaver, T. E., McIsaac, D. I., et al. (2025) · Anesthesiology · doi:10.1097/aln.0000000000005607 · Cited by 8
Controlled ovarian stimulation protocols for assisted reproduction: a network meta-analysis
Melo, P., Eapen, A. K., Chung, Y., Jeve, Y. B., Price, M. J., Sunkara, S. K., Macklon, N., Khalaf, Y., Tobı́as, A., Broekmans, F. J., et al. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012586.pub2 · Cited by 10
Sabeti, M. A., Harouni, A., & Gabbay, J. (2025) · Journal of Endodontics · doi:10.1016/j.joen.2025.08.002 · Cited by 8
Kılıç, M. E., Arayıcı, M. E., Yılancıoglu, R. Y., Turan, O. E., Özcan, E. E., & Yılmaz, M. B. (2025) · Heart · doi:10.1136/heartjnl-2025-326174 · Cited by 8
“Evidence certainty was graded with GRADE (Grading of Recommendations Assessment, Development and Evaluation), 18 with downgrades for risk of bias (serious ROBINS-I predominance), inconsistency (I²>50% or prediction interval (PI) spanning 1.0), indirectness (heterogeneous exposure/outcome…”
InfluentialSmoking cessation for people with severe mental illness: systematic review and network meta-analysis
Bekkering, G. E., Detraux, J., López-López, J. A., Hert, M. D., Evins, A. E., Vanhove, A.-C., Cloetens, H., Algarni, A. M., Vandendriessche, T., Agarwal, A., et al. (2026) · BMJ Medicine · doi:10.1136/bmjmed-2025-002190 · Cited by 8
“We used the GRADE approach for network meta-analysis 58 59 to assess the certainty of direct, indirect, and network estimates for all pooled outcomes.”
Welch, V. A., Hossain, A., Ghogomu, E. A. T., Riddle, A. Y., Cousens, S. N., Gaffey, M. F., Arora, P., Black, R. E., Bundy, D. A. P., Castro, M. C., et al. (2019) · Journal of Development Effectiveness · doi:10.1080/19439342.2019.1691627 · Cited by 8
“placebo, and assessed GRADE certainty for each outcome (Puhan et al. 2014).”
BackgroundMethodYang, M. G., Yu, Y., Lei, P., & Yuan, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-027376 · Cited by 8
He, H., Wang, X. D., Xiao, Y., Zheng, J., Wang, J., & Zhang, B. (2020) · Medicine · doi:10.1097/md.0000000000023747 · Cited by 8
“We will assess the quality of evidence regarding bias risk, indirectness, inconsistency, imprecision, and publication bias.([28])”
MethodSamacá-Samacá, D., Prieto‐Pinto, L., Yepes, A., Valderrama, C., & Hernández, F. (2023) · Lung Cancer Management · doi:10.2217/lmt-2022-0018 · Cited by 8
Hong, H., Friedland, A. E., Hu, M., Anstrom, K. J., Halabi, S., McKinnon, J. E., Amaravadi, R. K., Rojas‐Serrano, J., Abella, B. S., Portillo-Vasquez, A. M., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-065305 · Cited by 8
“19”
Tan‐Lim, C. S. C., Sajo, M. E. J., ORTEZA, K. E. M. P., Fernandez, P. B. A., & Vila, M. J. C. (2023) · Pediatric Allergy and Immunology · doi:10.1111/pai.14022 · Cited by 8
Determining the Optimal Treatment for Idiopathic Clubfoot
Wang, Y.-Y., Su, Y.-C., Tu, Y., Fang, C., Hong, C., Huang, M.-T., Lin, Y., Hsieh, M., Kuan, F., Shih, C., et al. (2023) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.22.01210 · Cited by 8
Comim, L. D., Marquezan, P. K., Knorst, J. K., Zanatta, F. B., Zenkner, J. E. D. A., & Alves, L. S. (2023) · Community Dentistry And Oral Epidemiology · doi:10.1111/cdoe.12933 · Cited by 10
Løvsletten, P. O., Wang, X., Pitre, T., Ødegaard, M., Veroniki, A. A., Lunny, C., Tricco, A. C., Agoritsas, T., & Vandvik, P. O. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111445 · Cited by 7
Torres, D., Zaror, C., Iturriaga, V., Tobı́as, A., & Brignardello‐Petersen, R. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13154557 · Cited by 9
“We used the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) working group’s approach for rating the certainty of the NMA evidence for all of the comparisons and outcomes as high, moderate, low, or very low [19,20].”
MethodSelf-citationComparative efficacy of nonpharmaceutical therapy in the treatment of dysphagia after stroke
Qin, W., Wang, Z., Zhong, Y., Yuan, Q., Jiang, X., Gao, J., Wu, J., & Zhang, Y. (2020) · Medicine · doi:10.1097/md.0000000000019115 · Cited by 8
“The quality evaluation of this study will be completed using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) established by the the World Health Organization.”
MethodRelationship between HLA-DRB1 gene polymorphism and breast cancer
Liu, L., Sun, X., Yuan, C., & Liu, H. (2021) · Medicine · doi:10.1097/md.0000000000025078 · Cited by 8
Stevenson, B., Tesfaye, W. H., Christenson, J. K., Mathew, C., Abrha, S., Peterson, G. M., Samarawickrema, I., & Thomas, J. (2021) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2021-001129 · Cited by 8
Tziatzios, G., Papaefthymiou, A., Facciorusso, A., Papanikolaou, I. S., Antonelli, G., Marco, S., Frazzoni, L., Fuccio, L., Paraskeva, K. D., Hassan, C., et al. (2022) · Digestive and Liver Disease · doi:10.1016/j.dld.2022.10.011 · Cited by 8
Ba, H., Zhu, F., Zhang, X., Mei, Z., & Zhu, Y. (2023) · Therapeutic Advances in Medical Oncology · doi:10.1177/17588359221148918 · Cited by 8
Li, R., Xu, C., Zhong, P.-Y., Wang, K., Luo, Y., Xiao, L., Dai, X., Han, J., & Zhang, X. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1181160 · Cited by 8
“Besides, we applied Grades of Recommendations Assessment, Development and Evaluation (GRADE) to evaluate the quality of outcomes according to direct, indirect and network comparison, respectively (Puhan et al., 2014; Salanti et al., 2014).”
MethodWang, T., Wang, J., Zhang, M., Zhang, H., Zhang, Q., Liu, G., Dong, W., Wang, Y., & Ji, B. (2024) · BMC Anesthesiology · doi:10.1186/s12871-024-02414-y · Cited by 8
“The quality of each direct, indirect, and NMA estimate was rated based on the four-step approach suggested by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group [21].”
MethodLou, S., Lv, H., Chen, Y., Zhang, L., & Tang, P. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012802 · Cited by 8
Zhang, L., Ningbo, L., Chang, R., Yang, C., Li, Q., Zuo, N., & Gu, Y. (2020) · Medicine · doi:10.1097/md.0000000000020689 · Cited by 8
“The quality of included studies will be assessed by grading of recommendations assessment development and evaluation (GRADE), and divided into 4 levels: high quality, moderate quality, low quality, and very low quality.([16])”
MethodAn, H., Zhao, J., Wang, J., Li, C., Jiang, Z., Yao, J., Zhang, X., & Wu, J. (2020) · Medicine · doi:10.1097/md.0000000000022734 · Cited by 8
Evidence inconsistency degrees of freedom in Bayesian network meta-analysis
Lin, L. (2020) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2020.1852247 · Cited by 8
Pharmacological interventions for acute hepatitis C infection
Kalafateli, M., Buzzetti, E., Thorburn, D., Davidson, B. R., Tsochatzis, E. A., & Gurusamy, K. S. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011644.pub3 · Cited by 11
“Then, we will present the estimates of the network meta-analysis and rate the quality of network meta-analysis eJect estimates as the best quality of evidence between the direct and indirect estimates (Puhan 2014).”
MethodYin, Z., Li, X., Wang, L., Sun, M., Zhao, L., & Liang, F. (2022) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2022/3288948 · Cited by 8
“+e Grading of Recommendations Assessment, Development, and Evaluation (GRADE) tool was used to assess the quality of evidence.”
MethodInfluentialManagement of post-traumatic stress disorder
Rehman, Y., Sadeghirad, B., Guyatt, G. H., McKinnon, M. C., McCabe, R. E., Lanius, R. A., Richardson, D. J., Couban, R., Sousa-Dias, H., & Busse, J. W. (2019) · Medicine · doi:10.1097/md.0000000000017064 · Cited by 8
“We will assess small study effects in direct comparisons using theHarbord test for dichotomous outcomes and the Egger test for continuous outcome when at least 10 studies are available.([62,64])”
MethodSelf-citationDayyeh, B. A., Murad, M. H., Bazerbachi, F., Buttar, N. S., Akshintala, V., Canto, M. I., & Chang, K. J. (2018) · Gastroenterology · doi:10.1053/j.gastro.2018.02.042 · Cited by 8
Self-citationWafa, I. A., Pratama, N. R., Budi, D. S., Sutanto, H., Rosyid, A. N., & Wungu, C. D. K. (2021) · medRxiv · doi:10.1101/2021.06.04.21258343 · Cited by 8
Kukendrarajah, K., Ahmad, M., Carrington, M., Ioannou, A., Taylor, J., Razvi, Y., Παπαγεωργίου, Ν., Mead, G. E., Nevis, I., D’Ascenzo, F., et al. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013255.pub2 · Cited by 8
Li, W., Han, H., Meng, J., Li, X., & Yao, Q.-P. (2024) · World Journal of Clinical Cases · doi:10.12998/wjcc.v12.i22.5083 · Cited by 7
Migliorati, M., Zuffanti, A., Capuano, M., Canullo, L., Caponio, V. C. A., & Menini, M. (2024) · International Orthodontics · doi:10.1016/j.ortho.2024.100957 · Cited by 8
Peng, Y., Liang, A., Chen, Z., Yang, B., Yu, W., Deng, J., Yu, F., Nie, Y., & Cheng, Y. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000002161 · Cited by 7
Maida, M. F., Facciorusso, A., Marasco, G., Calabrese, G., Ianiro, G., Jacques, J., Maselli, R., Hassan, C., Repici, A., Mitri, R. D., et al. (2025) · Digestive and Liver Disease · doi:10.1016/j.dld.2025.01.177 · Cited by 7
Wang, Y.-Y., Su, Y.-C., Lai, P., Chou, Y., Wu, P., Tsai, M., Tai, T., Wu, C., Chang, Y., Tu, Y., et al. (2025) · Archives of Osteoporosis · doi:10.1007/s11657-025-01515-6 · Cited by 7
“To assess the certainty of evidence (CoE) for treatment effect estimates, the GRADE methodology was utilized [25].”
Ding, Z., Gao, Z., Zhou, H., & Park, J. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-091743 · Cited by 9
“Ultimately, 10 studies were included in the meta-analysis (figure 1).”
Kim, S., Kim, M. S., Kwon, Y., Min, J.-S., ALRomi, A., Kim, J. Y., Kim, J., Shin, J. I., Yon, D. K., Chu, Y., et al. (2025) · Journal of the Korean Gastric Cancer Association · doi:10.5230/jgc.2025.25.e16 · Cited by 7
“We used a uniform analytic framework to enable a systematic evidence-level evaluation of 117 risk and protective factors associated with GC using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework [5,6].”
Jing, Y., Zhao, S., Guo, W., Qin, L., Li, Y., & You, D. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000002352 · Cited by 7
“The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was utilized to assess the certainty of the evidence using four levels (high, moderate, low, and very low) [ 32 ] .”
Li, X., Hu, Q., & Xu, T. (2025) · Expert Opinion on Drug Safety · doi:10.1080/14740338.2025.2502037 · Cited by 7
Zhao, Z., Zheng, Y., Song, X., Peng, C., Liao, S., Zhang, P., Tan, Y., Huang, X., & Zhang, L. (2025) · Clinical and Translational Allergy · doi:10.1002/clt2.70052 · Cited by 7
Liao, Y.-L., Hsu, P.-S., Lee, C.-T., Su, L., Shen, Y.-Y., Tsou, A., Lan, C., Tzeng, I., Liu, G., & Hsieh, P. (2025) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2025.1559388 · Cited by 7
Systematic Review and Network Meta-Analysis of Initial Treatments for Lupus Nephritis
Izcovich, A., Tortosa, F. G., Bengolea, A., Pissinis, M. M., Ragusa, M. A., Fielli, M., Agnoletti, C., Quintana, R., Malvar, A., Scolnik, M., et al. (2025) · Kidney International Reports · doi:10.1016/j.ekir.2025.06.047 · Cited by 7
Zhang, L., Wang, F., Tashiro, S., & Liu, P. J. (2025) · Clinical Nutrition · doi:10.1016/j.clnu.2025.08.005 · Cited by 7
Song, J., Dong, H., Wang, C., Zheng, Y., Huang, Y., Wei, Y., Zhang, K., Qing, H., Xiong, Z., Pan, Y., et al. (2026) · BMC Medicine · doi:10.1186/s12916-026-04682-6 · Cited by 8
“The certainty of evidence was evaluated using the CIN-eMA (Confidence in Network Meta-Analysis) framework, a GRADE-based approach designed for NMAs [32].”
Brigida, M., Crinò, S. F., Dell’Anna, G., Gadour, E., Almuhaidb, A., Donatelli, G., Maida, M. F., Spampinato, M. G., Stasi, E., Dell'Anna, A., et al. (2026) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2026.02.031 · Cited by 7
Shao, T., Li, X., Zhou, C., Zang, X., Malone, D. C., Zhang, L., Zhou, J., & Tang, W. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.651559 · Cited by 7
“Quality of Evidence Assessment We will evaluate the quality of evidence using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) evidence rating approach (38), which classifies the evidence as high, medium, low, and very low quality.”
MethodYin, S., Han, Y., Liu, Y., Chen, B., Fu, Z., Sheng, S., Wang, J., Shen, C., Wang, X., & Jia, Y. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.972498 · Cited by 7
“Finally, we used the network meta-analysis recommendations for grading and developed GRADE to assess the certainty of evidence (15).”
MethodWang, T., Li, C., Li, X., Zhai, J., Wang, S., & Shen, L. (2022) · European journal of medical research · doi:10.1186/s40001-022-00878-7 · Cited by 7
Zeng, B., Pan, H., Li, F., Ye, Z.-Y., Liu, Y., & Du, J. (2022) · Journal of Diabetes Investigation · doi:10.1111/jdi.13935 · Cited by 10
Long, C., Feng, H., Liu, Z., Li, Z., Liu, J., Jiang, Y., & Yue, R. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1028257 · Cited by 7
Bontinis, V., Bontinis, A., Koutsoumpelis, A., Claridge, M., Chorti, A. G., Rafailidis, V. D., Giannopoulos, A., & Ktenidis, K. (2023) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2023.05.029 · Cited by 8
Bin, C., Zhong, H., Mo, J., Wang, Z., Li, M., & Wei, S. (2023) · Journal of Cancer · doi:10.7150/jca.86043 · Cited by 7
“For direct evidence, we evaluate the quality from five aspects (risk of bias, inconsistency, inaccuracy, indirectness, imprecision and publication bias) [24].”
MethodZhou, J., Yang, M., Wang, F., Liu, S., Hei, M., & Jiang, M. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107601 · Cited by 7
Williams, M. J., Vogel, J. P., Gallos, I. D., Ramson, J. A., Chou, D., & Oladapo, O. T. (2023) · BMJ Global Health · doi:10.1136/bmjgh-2023-013109 · Cited by 7
“29 30 Since we undertook this process, further detailed guidance has been published providing practical strategies for GRADE-assessing NMA effect estimates, that seeks to reduce the significant workload involved.”
MethodHu, Q., Liu, X., Xiang, Y., Lei, X., Yu, H., Li, L., & Feng, J. (2024) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/bcpt.14043 · Cited by 7
Deng, L., Jiao, B., Cai, J., Xu, X., Chen, M., Yan, C., Zhu, T., Liu, J., Ma, D., & Chen, C. (2024) · BMC Anesthesiology · doi:10.1186/s12871-024-02743-y · Cited by 7
“The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) criteria was applied to evaluate the quality of evidence [30].”
MethodJia, M., Hu, F., & Yang, D. (2024) · European Journal of Pediatrics · doi:10.1007/s00431-024-05858-z · Cited by 7
“The GRADE approach can be used to assess the quality of evidence estimates in network meta-analysis (NMA) [26].”
Psychosocial interventions for psoriasis: a Bayesian network meta-analysis
Lu, L., Xu, Y., Shi, M., & Liu, A. (2025) · Journal of Dermatological Treatment · doi:10.1080/09546634.2024.2427321 · Cited by 7
Corticosteroids Versus Pentoxifylline for Severe Alcoholic Hepatitis
Njei, B. M., Do, A., McCarty, T. R., & Fortune, B. E. (2016) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000000585 · Cited by 7
“placebo 273 49 (10) 67/33 22 (7) — 62 (26) 17.”
InfluentialHussein, H., Zaccardi, F., Dhalwani, N. N., Davies, M. J., Khunti, K., & Gray, L. J. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-023206 · Cited by 7
“19 This approach will assess the quality of evidence for each outcome for direct and indirect estimates as well as rating the overall quality of evidence from the NMA effect estimates.”
BackgroundPharmacologic interventions for postoperative nausea and vomiting after thyroidectomy
Cho, Y., Choi, G. J., & Kang, H. (2019) · Medicine · doi:10.1097/md.0000000000014542 · Cited by 7
“the risk–benefit balance and a subsequent judgment on the strength of the recommendations.([1]) We will use a 4-step process:”
BackgroundMethodBusse, J. W., Craigie, S., Sadeghirad, B., Couban, R., Hong, P. J., Oparin, Y., May, C., Lok, A., & Guyatt, G. H. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024441 · Cited by 7
“We will also use the GRADE approach to assess quality of evidence for indirect and network (mixed) effect estimates.(41) 42 Indirect effect estimates are calculated from available ‘loops’ of evidence, which includes first-order loops (based on a single common comparator treatment, the difference between the treatments…”
MethodSelf-citationJohnston, B. W., Hill, R. A., Duarte, R. V., Chean, C. S., McAuley, D. F., Blackwood, B., Pace, N. L., & Welters, I. D. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1149-7 · Cited by 7
Anheyer, D., Klose, P., Koch, A. K., Haller, H., Dobos, G. J., & Cramer, H. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036050 · Cited by 7
Salman, S., Shehata, M., Ibrahim, A. M., Awad, M., Sarsik, S. M., El-Shirbiny, H., Fathi, M. M., Agha, N. A. Y., Anis, R., & Abdel‐Daim, M. M. (2021) · Dermatologic Therapy · doi:10.1111/dth.14793 · Cited by 7
“In terms of indirect evidence, we assessed certainty according to imprecision, intransitivity, and the lowermost-certainty ratings for the direct comparisons, forming the first-order loop with the most influencing to an indirect estimate.(20) To assess the certainty of the overall NMAs, direct and indirect evidence we…”
BackgroundYang, J., Wen, S., Kong, J., Yue, P., Cao, W., Xu, X., Zhang, Y., Chen, J., Liu, M., Fan, Y., et al. (2021) · Microbiology Spectrum · doi:10.1128/spectrum.00761-21 · Cited by 7
Zeng, X., Zhang, L., Jia, S., Lin, T., Liu, G., Yue, J., & Huang, X. (2021) · Anticancer Research · doi:10.21873/anticanres.15409 · Cited by 7
Lee, B., Kwon, C., & Jang, S. (2022) · Integrative Medicine Research · doi:10.1016/j.imr.2022.100832 · Cited by 7
Methods for managing miscarriage: a network meta-analysis
Gallos, I. D., Williams, H., Price, M. J., Eapen, A. K., Eyo, M. M., Tobı́as, A., Deeks, J. J., Tunçalp, Ö., Gülmezoglu, A. M., & Coomarasamy, A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012602 · Cited by 7
Ning, H., Yang, N., Ding, Y., Chen, H., Wang, L., Han, Y., Cheng, G., & Zou, M. (2022) · Medicina Clínica · doi:10.1016/j.medcli.2022.06.022 · Cited by 7
Abiramalatha, T., Ramaswamy, V. V., Bandyopadhyay, T., Somanath, S. H., Shaik, N. B., Kallem, V. R., Pullattayil, A. K., & Kaushal, M. (2022) · Journal of Neonatal-Perinatal Medicine · doi:10.3233/npm-221025 · Cited by 7
Phinyo, P., Krikeerati, T., Wongyikul, P., Lao‐Araya, M., & Thongngarm, T. (2022) · Asian Pacific Journal of Allergy and Immunology · doi:10.12932/ap-190822-1440 · Cited by 9
“27 The quality of evidence depends on study design, risk of bias, consistency, directness, and precision of the findings.”
Misra, S., Singh, S., Sarkar, S., Behera, B. K., & Jena, S. S. (2023) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000006578 · Cited by 7
Methodological quality of network meta-analysis in dentistry: a meta-research
Montagner, A. F., Angst, P. D. M., Raggio, D. P., van de Sande, F. H., & Tedesco, T. K. (2023) · Brazilian Oral Research · doi:10.1590/1807-3107bor-2023.vol37.0062 · Cited by 6
“85,86 This study has limitations, including a lack of consideration of the scope of the systematic reviews.”
BackgroundLiu, W., Sun, X., Gao, Y., Sun, H., Feng, H., Tan, H., Chen, Q., Peng, L., & Wu, X. (2023) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2023.105214 · Cited by 7
Medley, N., Vogel, J. P., Care, A., & Alfirević, Ž. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012505 · Cited by 8
Effects of 4 major brain protection strategies during aortic arch surgery
Fan, S., Wang, D., Wu, C., Pan, Z., Li, Y., An, Y., Li, H., Wang, G., Dai, J., & Wang, Q. (2018) · Medicine · doi:10.1097/md.0000000000011448 · Cited by 7
Pharmacological and non-pharmacological treatments for irritable bowel syndrome
Zhou, S., Liu, X., Wang, X., Xi, F., Luo, X., Yao, L., & Tang, H. (2019) · Medicine · doi:10.1097/md.0000000000016446 · Cited by 7
“0,([37]) which includes random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding of outcome assessment (detection bias), incomplete outcome data (attrition bias), selective reporting (reporting bias) and other bias.”
BackgroundMethodGeisler, B. P., Zahabi, L. E., Lang, A. E., Eastwood, N., Tennant, E., Lukić, L., Sharon, E., Chuang, H., Kang, C.-B., Clayton-Johnson, K., et al. (2021) · Systematic Reviews · doi:10.1101/2020.05.21.20109074 · Cited by 7
“Puhan MA, Schunemann HJ, Murad MH, Li T, Brignardello-Petersen R, Singh JA, Kessels AG, Guyatt GH: A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis.”
InfluentialHe, Z., Song, A., Zhang, Z., Zhang, Y., Liu, X., Zhao, L., Lv, X., Ren, G., & Li, Y. (2018) · Medicine · doi:10.1097/md.0000000000012096 · Cited by 7
Zhang, Y., Jiang, L., Wang, J., Wang, T., Chien, C., Huang, W., Fu, X., Xiao, Y., Fu, Q., Wang, S., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-1623050/v1 · Cited by 7
Acupoint injection in improving pain and joint function of knee osteoarthritis patients
Wang, H., Sun, J., Yu, X., & He, Y. (2021) · Medicine · doi:10.1097/md.0000000000024997 · Cited by 6
“We will use the grading of recommendation assessment, development and evaluation (GRADE) scoring method to grade the evidence of the outcome index.([21]) Based on randomized controlled trials, the “high” quality rating is the highest level of evidence.”
MethodDeclercq, I., Leontjevas, R., Verboon, P., VRIENDT, P. D., Gerritsen, D. L., & van Hooren, S. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05117-8 · Cited by 6
Yao, Y., Xu, S., Wang, T., & Jiang, R. (2024) · Medicine · doi:10.1097/md.0000000000039619 · Cited by 6
“[23] All disagreements of quality evaluation were resolved through discussion between all authors.”
BackgroundAlotaibi, F. F., Buti, J., Rocchietta, I., Nazari, N. S. M., Almujaydil, R., & D’Aiuto, F. (2025) · Clinical Oral Implants Research · doi:10.1111/clr.14435 · Cited by 6
“Certainty in the evidence (CiE) was assessed at the outcome level using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach (Puhan et al. 2014).”
Minozzi, S., Cinquini, M., Arienti, C., Battain, P. C., Brigadoi, G., Vicario, M. D., Domenico, G. D., Farma, T., Federico, S., Innocenti, T., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111811 · Cited by 6
Wu, X., Ding, Y., Cao, Q., Huang, J., Xu, X., Jiang, Y., Xu, Y., Lu, J., Xu, M., Wang, T., et al. (2025) · Nutrition Reviews · doi:10.1093/nutrit/nuaf056 · Cited by 5
Wang, J., Lian, N., Tang, K., Deng, Y., & Li, T. (2025) · Travel Medicine and Infectious Disease · doi:10.1016/j.tmaid.2025.102868 · Cited by 6
Self-citationBarbosa, L., Oliveira, V. M. R., Queiroz, I., Portilho, N. D. P., Furtado, W. J., Rodrigues, N. M. V., Guedes, K. M., Araújo, B., Itaya, E. D., Chagas, K. A., et al. (2025) · Journal of Interventional Cardiac Electrophysiology · doi:10.1007/s10840-025-02109-5 · Cited by 6
“…follows the GRADE framework but is specifically tailored for qualitative research, incorporat - ing assessments of methodological limitations (e.g., using the CAMELOT tool) and ensuring fidelity in reporting, ultimately guiding decision-makers with transparent and reliable evidence [21, 22].”
Moroni, F., Lamelas, P. M., Izcovich, A., Foroutan, F., Fremes, S. E., Alba, A. C., Agoritsas, T., Whitlock, R. P., Denicolai, M., Méndiz, O., et al. (2025) · Circulation Cardiovascular Interventions · doi:10.1161/circinterventions.125.015387 · Cited by 7
Deng, L., Zhao, C., Zhou, L., Mu, X., Sun, H., Li, L., & Zhang, Y. (2025) · Stem Cell Research & Therapy · doi:10.1186/s13287-025-04638-2 · Cited by 8
“Furthermore, when determining the final grade of the NMA results, careful consideration was given to local inconsistencies within the network and the precision of the combined effect size in the NMA [32].”
Lin, R., Hsu, C., Shih, M., Chien, K., & Wu, H. (2025) · Diabetes Obesity and Metabolism · doi:10.1111/dom.70224 · Cited by 6
Damen, J. A., Idema, D. L., Vernooij, R. W. M., Veld, L. F. H. I. ’., Kusters, M. P. T., Lokerse, M. E., de Kanter, E., Spijker, R., van der Braak, K., Jenniskens, K., et al. (2026) · Annals of Internal Medicine · doi:10.7326/annals-24-03764 · Cited by 7
Goel, N. N., Ferreyro, B. L., Pitre, T., Lewis, K. A., Homer-Bouthiette, C., Angriman, F., Kawano-Dourado, L. B., Drake, M. G., Eaton, T. L., Esbrook, C. L., et al. (2026) · American Journal of Respiratory and Critical Care Medicine · doi:10.1093/ajrccm/aamag302 · Cited by 12
Dehghani, M., Sadeghi, M., Barzkar, F., Maghsoomi, Z., Janani, L., Motevalian, S. A., Loke, Y. K., Ismail‐Beigi, F., Baradaran, H. R., & Khamseh, M. E. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1286827 · Cited by 9
Metadata
- Authors
- Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group
- Affiliations
- University of Zurich (CH)
- Mayo Clinic (US)
- Johns Hopkins University (US)
- University of Alabama (US)
- Maastricht University (NL)
- McMaster University (CA)
- Corresponding author
- Puhan, M. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2014-09-24
- Volume
- 349
- Issue
- sep24 5
- Pages
- g5630-g5630
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155798268
- Cited by
- 1,812
- References
- 39
- Citations by year
- 2026: 160 · 2025: 177 · 2024: 216 · 2023: 175 · 2022: 205 · 2021: 276 · 2020: 201 · 2019: 128 · 2018: 96 · 2017: 72 · 2016: 63 · 2015: 36 · 2014: 4
- FWCI
- 57.15
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-06-17)
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, GRADE, treatment effect estimates, comparative effectiveness
- MeSH terms
- Humans, Quality Assurance, Health Care, Meta-Analysis as Topic, Practice Guidelines as Topic, Evidence-Based Medicine, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review