The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
Open accessAbstract
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [45].”
From The Effect of Different Psychological Interventions on Athletes' Competitive Anxiety: A Systematic Review and Bayesian Network Meta-Analysis. (2026) · cites this paper as [45]
In the citing paper
“Therefore, we downgraded the quality of evidence for the comparison between SGLT2i and finerenone, SGLT2i and GLP-1 RA.”
In the citing paper
“Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head.”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
In the citing paper
“Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Items 701–800 of 1,001
Du, P., Jin, S., Lu, S., Wang, L., Ma, X., Wang, J., Huang, R., Luo, Q., Yang, S., & Feng, X. (2024) · Age and Ageing · doi:10.1093/ageing/afae035 · Cited by 15
Damen, J. A., Yang, B., Idema, D. L., Vernooij, R. W. M., Veld, L. F. H. I. ’., Kusters, M. P. T., Spijker, R., van der Braak, K., Heus, P., Jenniskens, K., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-00315 · Cited by 16
Network meta‐analysis in health care decision making
Rochwerg, B. N., Brignardello‐Petersen, R., & Guyatt, G. H. (2018) · The Medical Journal of Australia · doi:10.5694/mja18.00001 · Cited by 16
Self-citationRole of neoadjuvant chemotherapy in breast cancer patients: Systematic review and meta-analysis
Pathak, M., Deo, S. V. S., Dwivedi, S. N., Sreenivas, V., Thakur, B., Julka, P. K., & Rath, G. K. (2019) · Indian Journal of Medical and Paediatric Oncology · doi:10.4103/ijmpo.ijmpo_21_18 · Cited by 16
“[38] For every outcome, it emerged to be high grade except regarding two outcomes, DFS and BCS showing moderate grades.”
BackgroundShi, J., Xu, J., & Chen, Y. (2020) · Dermatologic Therapy · doi:10.1111/dth.13802 · Cited by 15
“The GRADE scoring system was used to evaluate the quality of the evidence in the effective evaluation of each result as high, medium, low, or very low.(17) RCTs were high-quality evidence at the beginning but maybe limited by one or more of the five categories: the risk of 2 of 12 SHI ET AL.”
MethodLin, S., Shi, Q., Yang, F., Wang, X., & Mao, J. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-037331 · Cited by 15
Chen, Z., Shi, Q., Peng, Y., Chen, Y., Cao, L., Pang, B., Ji, Z., Liu, C., & Zhang, J. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.652412 · Cited by 15
Ramaswamy, V. V., Abiramalatha, T., Bandyopadhyay, T., Shaik, N. B., Pullattayil, A. K., Cavallin, F., Roehr, C. C., & Trevisanuto, D. (2021) · Resuscitation · doi:10.1016/j.resuscitation.2021.10.027 · Cited by 15
Belavý, D. L., Diwan, A. D., Ford, J. J., Miller, C. T., Hahne, A. J., Mundell, N. L., Tagliaferri, S. D., Bowe, S. J., Pedder, H., Saueressig, T., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-057112 · Cited by 16
Costa, G. D. M., Ziegelmann, P. K., Zanatta, F. B., Martins, C. C., Costa, P. D. M., & Mello, C. F. (2022) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2022.110557 · Cited by 15
Marinelli, T. M., Davoudi, S., Foroutan, F., Orchanian‐Cheff, A., & Husain, S. (2022) · Transplant Infectious Disease · doi:10.1111/tid.13832 · Cited by 15
Ben‐Israel, D., Mann, J. A., Yang, M. M. H., Isaacs, A. M., Cadieux, M., Sader, N., Muram, S., Albakr, A. A., Manoranjan, B., Yu, R. W., et al. (2022) · Journal of Neurosurgery Pediatrics · doi:10.3171/2022.3.peds21512 · Cited by 15
Shao, T., Liang, L., Zhou, C., Tang, Y., Gao, W., Tu, Y., Yin, Y., Malone, D. C., & Tang, W. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1051581 · Cited by 19
“A recommended four-step approach was used in this study (35).”
MethodSaid, S. A., Alabed, S., Kaier, K., Tan, A. R., Bode, C., Meerpohl, J. J., & Duerschmied, D. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013252.pub2 · Cited by 17
Subramani, Y., Nagappa, M., Kumar, K., Mortuza, R., Fochesato, L.-A., Chohan, M. B. Y., Martin, J. E., Armstrong, K., & Singh, S. I. (2020) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2020.110102 · Cited by 15
Weibel, S., Jelting, Y., Pace, N. L., Rücker, G., Raj, D., Schaefer, M. S., Backhaus, I., Kienbaum, P., Eberhart, L. H. J., & Kranke, P. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012859 · Cited by 15
Ramaswamy, V. V., More, K., Roehr, C. C., Bandya, P., & Nangia, S. (2020) · doi:10.22541/au.159164046.64220644 · Cited by 15
The i-EUS consensus on EUS-guided gallbladder drainage: A 3-step modified Delphi approach
Facciorusso, A., Binda, C., Crinò, S. F., Lisotti, A., Spadaccini, M., Amato, A., Carrozza, L., Catena, F., Cobianchi, L., Coluccio, C., et al. (2025) · Endoscopic Ultrasound · doi:10.1097/eus.0000000000000128 · Cited by 14
Su, P., Li, M., Xiao, J., & Bai, L. (2025) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-025-06040-3 · Cited by 15
“Grading of recommendations, assessment, development, and evaluations (GRADE) was used to rate the evidence certainty for each outcome [29, 30].”
Jin, H., Xiang, Y., Feng, Y., Zhang, Y., Liu, S., Ruan, S., & Zhou, H. (2020) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2020/3237451 · Cited by 16
“*e Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system classifies the quality of evidence as 4 levels: high quality (十 十十十), moderate quality (十十十), low quality (十十), and very low quality (十) [18].”
BackgroundJalalzadeh, H., Wolfhagen, N., Harmsen, W. J., Griekspoor, M., & Boermeester, M. A. (2022) · Annals of Surgery Open · doi:10.1097/as9.0000000000000175 · Cited by 14
Busca, E., Airoldi, C., Bertoncini, F., Buratti, G., Casarotto, R., Gaboardi, S., Faggiano, F., Barisone, M., White, I. R., Allara, E., et al. (2022) · European Journal of Cardiovascular Nursing · doi:10.1093/eurjcn/zvac098 · Cited by 19
Rokkas, T., & Hold, G. L. (2023) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002519 · Cited by 14
Rokkas, T., Gisbert, J. P., Ekmektzoglou, K. A., Dassopoulos, T., Niv, Y., & O'Morain, C. A. (2024) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002751 · Cited by 14
“…Reporting Items for Systematic Reviews and Meta-Analyses) extension statement for interventions [22] whereas the quality of evidence derived from pairwise and NWM was achieved by using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) working group modality [23].”
MethodHe, R. X., Lin, F., Yu, B., & Huang, L. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e28776 · Cited by 14
Britten‐Jones, A. C., MacIsaac, R. J., Roberts, L. J., Kamel, J. T., Craig, J. P., Busija, L., & Downie, L. E. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-020804 · Cited by 17
“The strength and quality of the evidence for each outcome will be assessed using the Grading of Recommendations Assessment, Development and Evaluation approach.(70)”
MethodZou, K., Hu, J., Zhou, Q., Jiang, S., Dong, B., & Zhang, W. (2019) · Clinical Rheumatology · doi:10.1007/s10067-019-04704-0 · Cited by 14
Ferreyro, B. L., Angriman, F., Munshi, L., Sorbo, L. D., Ferguson, N. D., Rochwerg, B. N., Ryu, M., Saskin, R., Wunsch, H., da Costa, B. R., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01363-0 · Cited by 14
“Grading of recommendations We will rate the quality of each direct, indirect and NMA estimates based on the four-step approach suggested by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group.(40) The network effect estimates for each of the potential comparisons and every outco…”
MethodGangwani, M. K. (2021) · Annals of Gastroenterology · doi:10.20524/aog.2021.0602 · Cited by 14
“Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess the certainty of evidence (HIGH, MODERATE, LOW, VERY LOW) for each outcome was utilized [10].”
MethodZeng, H., Wu, Z., Yu, B., Wang, B., Wu, C., Wu, J., Lai, J., Gao, X., & Chen, J. (2021) · BMC Cancer · doi:10.1186/s12885-021-07973-8 · Cited by 14
“It was assumed that patients enrolled in all included studies were sampled from the same theoretical population [59, 60].”
BackgroundLuft, M. J., Dobson, E. T., Levine, A. A., Croarkin, P. E., & Strawn, J. R. (2021) · CNS Spectrums · doi:10.1017/s1092852921000377 · Cited by 15
Pharmacological and non-pharmacological treatments for insomnia
Wang, C., Deng, C., Han, Q., Zhu, D.-L., Li, Z.-M., & Li, J. (2021) · Medicine · doi:10.1097/md.0000000000026678 · Cited by 9
“0,([27]) which includes random sequence genera-”
Aziz, M., Khan, Z. H., Haghbin, H., Kamal, F., Sharma, S., Lee‐Smith, W., Pervez, A., Alastal, Y., Nawras, A. T., & Thosani, N. C. (2022) · Endoscopy International Open · doi:10.1055/a-1958-2348 · Cited by 14
“We used the “Grading of Recommendations Assessment, Development and Evaluation (GRADE)” approach to assess the certainty of evidence (High, Moderate, Low, Very Low) [8].”
MethodKhalifah, R. A. A., Flórez, I. D., Dennis, B. B., Neupane, B., Thabane, L., & Bassilious, E. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0105-4 · Cited by 14
“recommended by the GRADE working group [48].”
BackgroundMyocardial Protection With Different Cardioplegia in Adult Cardiac Surgery: A Network Meta-Analysis
Zhou, K., Zhang, X., Li, D., & Song, G. (2021) · Heart Lung and Circulation · doi:10.1016/j.hlc.2021.09.004 · Cited by 13
Nonpharmaceutical therapy for autism spectrum disorder
Jiang, X., Song, M. J., Qin, W., Jiang, X., Xu, X., & Yuan, Q. (2022) · Medicine · doi:10.1097/md.0000000000028811 · Cited by 13
Zhang, K., Wang, L., Zhang, D., Zhang, Z.-H., & Wang, H. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.799156 · Cited by 13
“The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) Working Group recommended a four-step evidence quality grading for network meta-analyses (21).”
MethodEisele‐Metzger, A., Schoser, D. S., Klein, M. D., Grummich, K., Schwarzer, G., Schwingshackl, L., Hermann, R. M., Biallas, B., Wilke, C., Meerpohl, J. J., et al. (2022) · Scandinavian Journal of Work Environment & Health · doi:10.5271/sjweh.4070 · Cited by 20
Zhang, M., Liang, Z., Li, Y., Meng, J., Jiang, X., Xu, B.-C., Li, H., & Liu, T. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1234017 · Cited by 17
Noori, A., Sadeghirad, B., Thabane, L., Bhandari, M., Guyatt, G. H., & Busse, J. W. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111276 · Cited by 14
Self-citationNetwork meta-analysis: The way forward for evidence-based decisions
Jaiswal, N., & Field, R. (2024) · Clinical Epidemiology and Global Health · doi:10.1016/j.cegh.2024.101531 · Cited by 13
Lee, C.-C., Chen, Y.-W., Kang, Y., Chen, J., Chen, C., Lu, C., Huang, T., & Gautama, M. S. N. (2024) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2024.104373 · Cited by 13
Burns, K. E. A., Sadeghirad, B., Ghadimi, M., Khan, J., Phoophiboon, V., Trivedi, V., Gomez-Builes, C., Giammarioli, B., Lewis, K. A., Chaudhuri, D., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04958-4 · Cited by 14
“We considered a minimally important difference of 3–4% to be important in network estimates [28–30].”
BackgroundBhandari, M., Samanta, J., Spadaccini, M., Fugazza, A., Crinò, S. F., Gkolfakis, P., Τriantafyllou, Κ., Dhar, J., Maida, M. F., Pugliese, N., et al. (2024) · Diagnostics · doi:10.3390/diagnostics14121238 · Cited by 13
Xu, Y., Liu, R., Huang, L., Qin, Y., Liu, W., Huang, S., & Zhang, J. (2024) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgae877 · Cited by 13
Chandan, S., Mohan, B. P., Khan, S. R., Bhogal, N., Canakis, A., Bilal, M., Dhaliwal, A., Aziz, M., Mashiana, H. S., Singh, S., et al. (2020) · Endoscopy International Open · doi:10.1055/a-1164-6398 · Cited by 13
“Quality of evidence The quality of evidence was rated for results from the meta-analysis according to the GRADE working group approach [26].”
MethodLytvyn, L., Zeraatkar, D., Anbari, A. B., Ginex, P., Zoratti, M. J., Niburski, K., Sadeghirad, B., Siedler, M. R., Thabane, L., & Morgan, R. L. (2020) · Oncology nursing forum · doi:10.1188/20.onf.e171-e189 · Cited by 13
Compression methods after femoral artery puncture
Chen, H.-Z., Liang, W., Yao, W.-F., & Liu, T.-X. (2021) · Medicine · doi:10.1097/md.0000000000024506 · Cited by 13
Tedesco, T. K., Calvo, A. F. B., Pássaro, A. L., Araújo, M. P., Ladewig, N. D. M., Scarpini, S., Lara, J. S., Braga, M. M., Gimenez, T., & Raggio, D. P. (2021) · Acta Odontologica Scandinavica · doi:10.1080/00016357.2021.1928748 · Cited by 13
How to Prevent Surgical Site Infection in Vascular Surgery: A Review of the Evidence
Zhao, A. H., Kwok, C. H. R., & Jansen, S. J. (2021) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2021.06.045 · Cited by 13
Kovalic, A. J. (2022) · Journal of Clinical and Experimental Hepatology · doi:10.1016/j.jceh.2022.01.011 · Cited by 14
Dai, Y., Fan, H., Hu, Y., Zhao, Z., & Liu, C. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.943063 · Cited by 13
Zhou, L.-P., Zhang, R., Shang, J., Kang, L., Zhang, Z., Zhang, B., Wang, J.-Q., Jia, C.-Y., Zhao, C.-H., Zhang, H.-Q., et al. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000798 · Cited by 14
Zhang, J., Li, R.-K., Chen, K., Ge, L., & Tian, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010264 · Cited by 13
Fluid resuscitation in critically ill patients: a systematic review and network meta-analysis
Liu, C., Mao, Z., Hu, P., Hu, X., Kang, H., Hu, J., Yang, Z., Ma, P., & Zhou, F. (2018) · Therapeutics and Clinical Risk Management · doi:10.2147/tcrm.s175080 · Cited by 15
“18 This approach classifies the strength of evidence as high, moderate, low, or very low.”
Ueyama, H. A., Kuno, T., Ando, T., Hayashida, K., & Takagi, H. (2020) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2020.02.088 · Cited by 13
Mohan, B. P., Canakis, A., Khan, S. R., Chandan, S., Ponnada, S., McDonough, S., & Adler, D. G. (2020) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000001377 · Cited by 13
Steeb, T., Wessely, A., Schmitz, L., Heppt, F., Kirchberger, M. C., Berking, C., & Heppt, M. V. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.06.021 · Cited by 13
“Four studies provided data on the lesion-specific clearance rate for an NMA (Figure 3c) (ALA-AK-CT010, personal communication; Kaufmann et al., 2008; Sotiriou et al., 2009; Szeimies et al., 2002).”
MethodInfluentialReporting of Cochrane systematic review protocols with network meta‐analyses—A scoping review
Wang, R., Dwan, K. M., Showell, M. G., van Wely, M., Mol, B. W., Askie, L. M., & Seidler, A. L. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1531 · Cited by 12
Nanji, K., Sarohia, G. S., Kennedy, K., Ceyhan, T., McKechnie, T., Phillips, M., Devji, T., Thabane, L., Kaiser, P. K., Sarraf, D., et al. (2021) · Ophthalmology · doi:10.1016/j.ophtha.2021.11.024 · Cited by 13
Moreno-Drada, J. A., Abreu, L. G., Lino, P. A., Martins, M. A. P., Pordeus, I. A., & de Abreu, M. H. N. G. (2021) · Journal of Cranio-Maxillofacial Surgery · doi:10.1016/j.jcms.2021.04.014 · Cited by 12
Wang, X., Sadeghirad, B., Morgan, R. L., Zeratkaar, D., Chang, Y., Crandon, H. N., Couban, R., Foroutan, F., & Flórez, I. D. (2022) · Scientific Reports · doi:10.1038/s41598-022-21318-w · Cited by 12
Huang, M., Zeng, B., Tseng, P., Hsu, C.-W., Wu, Y., Tu, Y., Stubbs, B., Carvalho, A. F., Liang, C., Chen, T., et al. (2023) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2023.06.013 · Cited by 16
Vilchez, G., Meislin, R., Lin, L., Gonzalez, K., McKinney, J. A., Kaunitz, A. M., Stone, J. L., & Sanchez‐Ramos, L. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2022.09.043 · Cited by 12
Martínez-Miranda, P., Jiménez‐Rejano, J., Rosales‐Tristancho, A., & Casuso-Holgado, M. J. (2023) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2023.102411 · Cited by 12
Lee, B., Kwon, C., Lee, H. W., Nielsen, A., Wieland, L. S., Kim, T., Birch, S., Alræk, T., & Lee, M. S. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e25650 · Cited by 13
“The certainty of evidence of NMA findings was assessed according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method [11,12].”
MethodChung, S.-M., Wang, J.-C., Lin, C., Liu, S.-C., Wu, P., Kuan, F., Fang, C., Tu, Y., Hsu, K., Lai, P., et al. (2024) · Regional Anesthesia & Pain Medicine · doi:10.1136/rapm-2023-105104 · Cited by 13
Carron, M., Tamburini, E., Linassi, F., Pettenuzzo, T., Boscolo, A., & Navalesi, P. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13072100 · Cited by 12
Meng, L., Zhao, X., Sun, Y., Cheng, S., Bao, L., Fang, K., Yu, Q. Z., Zheng, Y., Wang, J., Luo, M., et al. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.05.033 · Cited by 12
Crocker, T. F., Lam, N., Ensor, J., Jordão, M., Bajpai, R. C., Bond, M., Förster, A., Riley, R. D., Andre, D., Brundle, C., et al. (2023) · Health Technology Assessment · doi:10.3310/hnrp2514 · Cited by 13
Khalili, M., Sadeghirad, B., Bach, P., Crabtree, A., Javadi, S., Sadeghi, E., Moradi, S., Fashami, F. M., Nakhaeizadeh, M., Salehi, S., et al. (2024) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-024-01379-w · Cited by 15
“We used the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach to assess the certainty of evidence for direct, indirect, and network estimates across all outcomes (Brignardello-Petersen et al., 2018; Puhan et al., 2014).”
Comparative efficacy of different bowel preparations for colonoscopy: A network meta-analysis.
Maida, M. F., Marasco, G., Fuccio, L., Vitello, A., Mocciaro, F., Amata, M., Fabbri, A., Mitri, R. D., Vassallo, R., Ramai, D., et al. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2024.11.019 · Cited by 12
Zeraatkar, D., Ling, M., Kirsh, S., Jassal, T., Pitre, T., Chakraborty, S. P., Turner, T., Turkstra, L. S., McIntyre, R. S., Izcovich, A., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-086407 · Cited by 15
Janitra, F. E., Chen, R., Sung, C., Wang, C., Wibawa, Y. A., Banda, K. J., Chiang, K., & Chou, K. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105151 · Cited by 12
Song, K., Liang, J., Zhang, M., Cai, S., Wang, Y., & Wu, W. (2025) · BMC Surgery · doi:10.1186/s12893-025-02992-9 · Cited by 17
“Additionally, the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) [15] approach was tailored for network meta-analysis and used to appraise the overall certainty of the evidence regarding each outcome (Supplementary Table 2).”
Treatment for gastrointestinal and pancreatic neuroendocrine tumours: a network meta-analysis
Walter, M. A., Nesti, C., Spanjol, M., Kollár, A., Bütikofer, L., Gloy, V., Dumont, R. A., Seiler, C. A., Christ, E., Radojewski, P., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013700.pub2 · Cited by 12
Mishra, A., Maiti, R., Mishra, B. R., & Jena, M. (2021) · Pharmacological Research · doi:10.1016/j.phrs.2021.105478 · Cited by 14
Lunny, C., Tricco, A. C., Veroniki, A. A., Dias, S., Hutton, B., Salanti, G., Wright, J. M., White, I. R., & Whiting, P. F. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045987 · Cited by 13
Light, A., Jabarkhyl, D., Gilling, P. J., George, G., Hemelrijck, M. V., Challacombe, B. J., Malde, S., Popert, R., Prokar Dasgupta, A., & Elhage, O. (2021) · European Urology · doi:10.1016/j.eururo.2021.04.012 · Cited by 12
Adams, A., Scheckel, B., Habsaoui, A., Haque, M., Kuhr, K., Monsef, I., Bohlius, J., & Skoetz, N. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012633.pub2 · Cited by 12
Lee, B., & Kwon, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192012994 · Cited by 13
“, NMA) effect estimates derived for our primary and secondary outcomes was evaluated using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach [20].”
MethodKovalic, A. J., Gozar, M., Da, B. L., Bernstein, D., & Satapathy, S. K. (2022) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000002463 · Cited by 13
Chang, C., Tu, Y., Kao, M., Shih, P.-C., Su, I.-M., Lin, H., Chien, Y.-J., Wu, M., Chen, C., & Chen, C.-T. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101787 · Cited by 12
Tokavanich, N., Prasitlumkum, N., Kewcharoen, J., Chokesuwattanaskul, R., Phannajit, J., Cheungpasitporn, W., AKOUM, N. W., Ranjan, R., Bunch, T. J., & Navaravong, L. (2023) · Journal of Cardiovascular Electrophysiology · doi:10.1111/jce.15831 · Cited by 12
“The quality of treatment effects estimation was rated according to a GRADE working group approach.(21) Brooks–Gelman Rubin diagnostic was performed to assess the convergence of models(22) (Online Supporting Information Data S2, S5, and S9).”
MethodKim, M. S., Kim, J. H., Ryu, S., Lee, S. W., Yon, D. K., Kim, E. Y., Koyanagi, A., Dragioti, E., Shin, J. I., & Smith, L. (2023) · World Journal of Pediatrics · doi:10.1007/s12519-023-00716-8 · Cited by 12
“To evaluate the certainty of the evidence, we applied the Grading of Recommendations, Assessments, Developments, and Evaluation (GRADE) approach [15].”
Yang, T. X., Tan, A. Y., Leung, W. H., Chong, D., & Chow, Y. F. (2023) · Cureus · doi:10.7759/cureus.38238 · Cited by 12
“The Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidelines were used to assess the quality of evidence of each network [27].”
MethodSiribumrungwong, B., Srikuea, K., Orrapin, S., Benyakorn, T., Rerkasem, K., & Thakkinstian, A. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024813 · Cited by 12
“Grading evidence Quality of evidence will be graded separately for each outcome using a tool suggested by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group.(34) Five domains will be assessed including study limitations, consistency, indirectness, imprecision and publication bi…”
Lewis, K. A., Chaudhuri, D., Guyatt, G. H., Burns, K. E. A., Bosma, K. J., Ge, L., Karachi, T., Piraino, T., Fernando, S. M., Ranganath, N., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030407 · Cited by 12
Self-citationFragility index of network meta-analysis with application to smoking cessation data
Xing, A., Chu, H., & Lin, L. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.003 · Cited by 12
“Different sources of evidence may have different extents of credibility [74, 75]; event status modifications in comparisons from difference sources of evidence may not have an equivalent role.”
BackgroundAl-Zoobaee, F. W. A., Loo, Y. S., Veettil, S. K., Gopinath, D., Maharajan, M. K., & Menon, R. K. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17238891 · Cited by 12
“Additionally, to assess the degree of confidence or certainty of our findings, we used the GRADE approach, which is the most recommended approach for the evolution of strength of evidence [77].”
MethodWong, C. H. L., Cheung, W. K. W., Zhong, C. C., Yeoh, E., Hung, C. T., Yip, B. H. K., Wong, E. L., Wong, S. Y. S., & Chung, V. C. (2021) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.103904 · Cited by 12
Wu, Y., Dai, Y., Zhang, L., Pan, H.-G., Chen, W., Li, R., & Hu, L. (2021) · PLoS ONE · doi:10.1371/journal.pone.0255665 · Cited by 14
Battle, C. E., Abdul‐Rahim, A. H., Shenkin, S. D., Hewitt, J., & Quinn, T. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013306 · Cited by 12
Fountain, D. M., Jenkinson, M. D., Bryant, A., Vale, L. D., Bulbeck, H., Hart, M. G., & Barone, D. G. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013630 · Cited by 12
Oh, S., Purja, S., Shin, H., Kim, M. S., Park, S., Kronbichler, A., Smith, L., Eisenhut, M., Shin, J. I., & Kim, E. Y. (2022) · Vaccines · doi:10.3390/vaccines10101572 · Cited by 12
Treatment of severe covid-19 with interleukin 6 receptor inhibition
Rajasundaram, S., Burgess, S., & Gill, D. (2022) · BMJ Medicine · doi:10.1136/bmjmed-2022-000144 · Cited by 12
Tang, Q., Zhong, Y., Xu, C., Li, W., Wang, H., & Hou, Y. (2022) · PubMed Central · doi:10.1097/md.0000000000029126 · Cited by 11
“Finally, we appraisedthequalityofevidenceforalloutcomesusingaframework designed explicitly by the Grading of Recommendations Assessment,Development,andEvaluationworkinggroupfor randomized studies in the context of a network meta-analysis.([15])”
MethodLin, F., Yu, B., Deng, B., & He, R. X. (2023) · Medicine · doi:10.1097/md.0000000000034746 · Cited by 11
“The overall quality of the evidence was assessed using the Grading of Recommendations, Assessment, Development, and Evaluation tool.[14] The risk of bias in each included study was evaluated using the Cochrane Collaboration bias assessment tool.”
MethodTreatment of diabetic kidney disease. A network meta-analysis
Büttner, F., Barbosa, C. V., Lang, H., Tian, Z., Melk, A., & Schmidt, B. M. W. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293183 · Cited by 13
“Moreover, we rated the quality of evidence of each pairwise comparison according to GRADE (Grading of Recommendations Assessment, Development and Evaluation) [12].”
Wang, J., Li, X., Yang, F., Guo, P., Ren, C., Duan, Z., & Zhang, Y. (2024) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-024-05072-5 · Cited by 11
“The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system was employed to evaluate the quality of evidence for the outcomes in the meta-analysis [40].”
Method
Metadata
- Authors
- Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group
- Affiliations
- University of Zurich (CH)
- Mayo Clinic (US)
- Johns Hopkins University (US)
- University of Alabama (US)
- Maastricht University (NL)
- McMaster University (CA)
- Corresponding author
- Puhan, M. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2014-09-24
- Volume
- 349
- Issue
- sep24 5
- Pages
- g5630-g5630
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155798268
- Cited by
- 1,812
- References
- 39
- Citations by year
- 2026: 160 · 2025: 177 · 2024: 216 · 2023: 175 · 2022: 205 · 2021: 276 · 2020: 201 · 2019: 128 · 2018: 96 · 2017: 72 · 2016: 63 · 2015: 36 · 2014: 4
- FWCI
- 57.15
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-06-17)
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, GRADE, treatment effect estimates, comparative effectiveness
- MeSH terms
- Humans, Quality Assurance, Health Care, Meta-Analysis as Topic, Practice Guidelines as Topic, Evidence-Based Medicine, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review