The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
Open accessAbstract
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [45].”
From The Effect of Different Psychological Interventions on Athletes' Competitive Anxiety: A Systematic Review and Bayesian Network Meta-Analysis. (2026) · cites this paper as [45]
In the citing paper
“Therefore, we downgraded the quality of evidence for the comparison between SGLT2i and finerenone, SGLT2i and GLP-1 RA.”
In the citing paper
“Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head.”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
In the citing paper
“Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Items 401–500 of 1,001
Günhan, B. K., Friede, T., & Held, L. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1285 · Cited by 35
Silva, F. F. V. E., Chauca-Bajaña, L., Caponio, V. C. A., Cueva, K. A. S., Ron, B. V., Padín-Iruegas, E., Almeida, L. L., Lorenzo‐Pouso, A. I., Peñaranda, J. M. S., & Pérez‐Sayáns, M. (2024) · Odontology · doi:10.1007/s10266-024-00949-7 · Cited by 42
Roberts, D. A., Best, L. M., Freeman, S. C., Sutton, A. J., Cooper, N. J., Arunan, S., Begum, T., Williams, N. R., Walshaw, D., Milne, E., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013155.pub2 · Cited by 33
Pratt, M., Wieland, L. S., Ahmadzai, N., Butler, C., Wolfe, D. M., Pussagoda, K., Skidmore, B., Veroniki, A. A., Rios, P. D. L., Tricco, A. C., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01328-3 · Cited by 33
Lindson, N., Theodoulou, A., Livingstone‐Banks, J., Aveyard, P. N., Fanshawe, T. R., Ordóñez-Mena, J. M., Sutton, A. J., Freeman, S. C., Agrawal, S., & Hartmann‐Boyce, J. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015226 · Cited by 33
Kanie, T., Mizuno, A., Yoneoka, D., Tam, W., Morze, J., Rynkiewicz, A., Xin, Y., Wu, O., & Kwong, J. S. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013650 · Cited by 34
Tseng, P., Chen, Y., Zeng, B., Zeng, B., Hung, C., Sun, C., Cheng, Y., Stubbs, B., Carvalho, A. F., Brunoni, A. R., et al. (2023) · Alzheimer s Research & Therapy · doi:10.1186/s13195-023-01164-2 · Cited by 33
“We evaluated the quality of evidence with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) rating tools described previously [56, 57].”
MethodLaguna, J. M., Puhan, M. A., Rodríguez‐Artalejo, F., Pauw, R. D., Wyper, G. M. A., Devleesschauwer, B., Santos, J. V., & Hincapié, C. A. (2023) · International Journal of Public Health · doi:10.3389/ijph.2023.1605763 · Cited by 33
“Beyond the wide use of GRADE in systematic reviews and clinical practice guidelines, important variations have emerged for other types of evidence [14], including modelling studies [15].”
BackgroundSelf-citationYang, M. G., He, M., Zhao, M., Zou, B., Liu, J. L., Luo, L.-M., Li, Q., He, J., & Lei, P. (2017) · Current Medical Research and Opinion · doi:10.1080/03007995.2017.1281110 · Cited by 33
Pettenuzzo, T., Boscolo, A., Pistollato, E., Pretto, C., Giacon, T. A., Frasson, S., Carbotti, F. M., Medici, F., Pettenon, G., Carofiglio, G., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04924-0 · Cited by 32
“…indirectness, publication bias, intransitivity, incoherence, and imprecision, was used to assess the certainty of evidence related to the each of the outcomes [14] Imprecision for each comparison was incorporated only at the network level, not at the level of the direct or indirect estimate.”
MethodLi, Y.-K., Li, W., Ren, H., Xiao, C., Guo, Z., & Luo, J. (2025) · Inflammopharmacology · doi:10.1007/s10787-025-02044-x · Cited by 32
“We applied the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group approach to evaluate the certainty of evidence (Guyatt et al. 2011; Puhan et al. 2014).”
Khouri, C., Jouvé, T., Blaise, S., Carpentier, P., Cracowski, J., & Roustit, M. (2016) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.12980 · Cited by 32
Singh, J. A., Hossain, A., Kotb, A., & Wells, G. A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0328-z · Cited by 32
“The quality of evidence was assessed as recommended in a recent study [32].”
BackgroundSelf-citationControlled ovarian stimulation protocols for assisted reproduction: a network meta-analysis
Gallos, I. D., Eapen, A. K., Price, M. J., Sunkara, S. K., Macklon, N., Bhattacharya, S., Khalaf, Y., Tobı́as, A., Deeks, J. J., Rajkhowa, M., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012586 · Cited by 32
Roccarina, D., Best, L. M., Freeman, S. C., Roberts, D. A., Cooper, N. J., Sutton, A. J., Benmassaoud, A., Torres, M. C. P., Prat, L. I., Csenar, M., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013121.pub2 · Cited by 34
Efficacy and Safety of Antibiotics for Treatment of Scrub Typhus
Yang, J., Luo, L., Chen, T., Li, L., Xu, X., Zhang, Y., Cao, W., Yue, P., Bao, F., & Liu, A. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.14487 · Cited by 32
Lai, C.-C., Chen, S., Tu, Y., Ding, Y., & Lin, J. (2021) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2021.103276 · Cited by 32
Zhang, C., Kwong, J. S. W., Yuan, R.-X., Chen, H., Xu, C., Wang, X., Yang, G.-L., Yan, J.-Z., Peng, L., Zeng, X., et al. (2017) · Scientific Reports · doi:10.1038/srep41021 · Cited by 32
Wen, D., Tu, R., Flitcroft, I., Wang, Q., Huang, Y., Song, B., Yu, A., Hu, L., Zhao, Y., Bao, F., et al. (2018) · Journal of Refractive Surgery · doi:10.3928/1081597x-20180905-01 · Cited by 32
Grant, S., Azhar, G., Han, E., Booth, M., Motala, A., Larkin, J., & Hempel, S. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003822 · Cited by 31
Fan, S., Li, H., Wang, D., Wu, C., Pan, Z., Li, Y., An, Y., Wang, G., Dai, J., & Wang, Q. (2019) · International Journal of Surgery · doi:10.1016/j.ijsu.2019.01.009 · Cited by 31
Bi, L.-W., Yan, B.-L., Yang, Q., Li, M., & Cui, H. (2019) · Medicine · doi:10.1097/md.0000000000017521 · Cited by 31
“Discussion The intestine of the preterm infants easily trended to colonized by pathogenic bacteria in the neonatal intensive care units (NICUs), which may be because of the delayed breastfeeding, early antibiotic intervention, and/or total parenteral nutrition.([20]) This contributed to the higher incidence of NEC and…”
BackgroundMethodJiang, N., Rao, F., Xiao, J., Yang, J., Wang, W., Li, Z., Huang, R., Liu, Z., & Guo, T. (2020) · International Journal of Surgery · doi:10.1016/j.ijsu.2020.07.066 · Cited by 31
Using GRADE evidence to decision frameworks to choose from multiple interventions
Piggott, T., Brożek, J., Nowak, A. J., Dietl, H., Dietl, B., Saz‐Parkinson, Z., Mirzayev, F., González-Angulo, L., Yepes-Núñez, J. J., Mustafa, R. A., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.10.016 · Cited by 31
Self-citationXie, P.-W., Yan, J., Ye, L., Wang, C., Li, Y., Chen, Y., & Li, G. (2021) · Surgical Endoscopy · doi:10.1007/s00464-021-08386-1 · Cited by 31
Antón-Vázquez, V., Hine, P., Krishna, S., Chaplin, M., & Planche, T. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013235.pub2 · Cited by 31
Progestogens for preventing miscarriage: a network meta-analysis
Devall, A., Papadopoulou, A., Haas, D. M., Price, M. J., Coomarasamy, A., & Gallos, I. D. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013792 · Cited by 31
Direct materials for restoring caries lesions
Pilcher, L., Pahlke, S. C., Urquhart, O., O’Brien, K. K., Dhar, V. K., Fontana, M., González‐Cabezas, C., Keels, M. A., Mascarenhas, A. K., Nascimento, M. M., et al. (2023) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2022.09.012 · Cited by 34
Tseng, P., Chiu, H., Suen, M., Zeng, B., Wu, M., Tu, Y., Hung, K., Wu, Y., Su, K., Li, D., et al. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115316 · Cited by 31
Ng, S. S., Lai, N. M., Nathisuwan, S., Jahan, N. K., Dilokthornsakul, P., Kongpakwattana, K., Hollingworth, W., & Chaiyakunapruk, N. (2020) · Scientific Reports · doi:10.1038/s41598-019-57370-2 · Cited by 30
deSouza, I. S., Thode, H. C., Shrestha, P., Allen, R. W., Koos, J. A., & Singer, A. J. (2021) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2021.11.011 · Cited by 33
Torres, M. C. P., Best, L. M., Freeman, S. C., Roberts, D. A., Cooper, N. J., Sutton, A. J., Roccarina, D., Benmassaoud, A., Prat, L. I., Williams, N. R., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013122.pub2 · Cited by 30
Huang, Y., Fan, H., Li, N., & Du, J. (2019) · Cancer Medicine · doi:10.1002/cam4.2104 · Cited by 30
T-piece resuscitator or self-inflating bag during neonatal resuscitation: a scoping review
Roehr, C. C., Davis, P. G., Weiner, G. M., Wyllie, J. P., Wyckoff, M. H., & Trevisanuto, D. (2020) · Pediatric Research · doi:10.1038/s41390-020-1005-4 · Cited by 30
Dreweck, F. D. S., Burey, A., Dreweck, M. D. O., Loguércio, A. D., & Reis, A. (2021) · Clinical Oral Investigations · doi:10.1007/s00784-021-03844-5 · Cited by 30
Progress and challenges of network meta‐analysis
Tian, J., Gao, Y., Zhang, J., Yang, Z., Dong, S., Zhang, T., Sun, F., Wu, S., Wu, J., Wang, J., et al. (2021) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12443 · Cited by 29
Kiss, S., Gede, N., Soós, A., Hegyi, P. J., Nagy, B., Imrei, M., Czibere, B., Farkas, N., Hanák, L., Szakács, Z., et al. (2021) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2021.103504 · Cited by 31
Singh, N. P., Monks, D., Makkar, J. K., Palanisamy, A., Sultan, P., & Singh, P. M. (2021) · Anaesthesia · doi:10.1111/anae.15645 · Cited by 30
“confidence in network meta-analysis (CINeMA) approach [11, 12].”
Cheng, Y., Zeng, B., Hung, C., Su, K., Wu, Y., Tu, Y., Lin, P., Stubbs, B., Carvalho, A. F., Liang, C., et al. (2021) · The Journal of Headache and Pain · doi:10.1186/s10194-022-01401-3 · Cited by 32
“To be specific, we evaluated the GRADE ratings according to the rationale of the articles published in the BMJ [57] and the Lancet [58].”
MethodFacciorusso, A., Mangiavillano, B., Paduano, D., Binda, C., Crinò, S. F., Gkolfakis, P., Ramai, D., Fugazza, A., Tarantino, I., Lisotti, A., et al. (2022) · Cancers · doi:10.3390/cancers14133291 · Cited by 30
“Clinical success was defined by the decrease in bilirubin levels to <50% as compared to baseline within 7 days in 3 RCTs [14,15,17], within 14 days in 1 RCT [10], with no specific timespan in 1 RCT [18].”
BackgroundMethodXiao, C., Ren, H., Chen, H., Liu, W.-H., Luo, Z., Li, W., & Luo, J. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1145456 · Cited by 29
“Besides, we employed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group approach to assess the strength of evidence (Guyatt et al., 2011; Puhan et al., 2014).”
MethodMaida, M. F., Marasco, G., Maas, M. H. J., Ramai, D., Spadaccini, M., Sinagra, E., Facciorusso, A., Siersema, P. D., & Hassan, C. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2024.09.003 · Cited by 29
Rajabinejad, M., Ranjbar, S., Hezarkhani, L. A., Salari, F., Karaji, A. G., & Rezaiemanesh, A. (2019) · Journal of Cellular Physiology · doi:10.1002/jcp.29401 · Cited by 29
“For clinical results that were not homogenous enough for meta‐ analyses, we used the Grading of Recommendations, Assessment, Development, and Evaluation model (via GRADpro/GTD online software) to access the quality of the evidence (Puhan et al., 2014).”
MethodSurgical Treatment of Peripheral Vascular Malformations: A Systematic Review and Meta-Analysis
Lokhorst, M. M., Jolink, F., Horbach, S. E. R., Spuls, P. I., & van der Horst, C. M. A. M. (2021) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000007837 · Cited by 29
Wang, S., Hung, Y., Hsu, C.-C., Hu, C.-H., Huang, R.-Y., Sung, C., Li, Y., Kou, H.-W., Chen, M.-Y., Chang, S.-C., et al. (2021) · Nutrients · doi:10.3390/nu13114049 · Cited by 29
Wu, Y., Shen, S.-F., Lee, S., Chen, L., & Tung, H. (2023) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2023.104949 · Cited by 32
Ballesio, A., Zagaria, A., Vacca, M., Pariante, C. M., & Lombardo, C. (2023) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2023.05.006 · Cited by 30
“The certainty of the evidence pertaining to the primary outcomes was assessed through the grading of recommendations assessment, development, and evaluation (GRADE) approach for NMA, as outlined by Puhan et al. (2014).”
Pharmacological interventions for alcoholic liver disease (alcohol-related liver disease)
Buzzetti, E., Kalafateli, M., Thorburn, D., Davidson, B. R., Thiele, M. S., Gluud, L. L., Giovane, C. D., Askgaard, G., Krag, A. A., Tsochatzis, E. A., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011646.pub2 · Cited by 30
White, L., Reardon, D., Davis, K., Velli, G., & Bright, M. R. (2021) · Journal of Anesthesia · doi:10.1007/s00540-021-03000-z · Cited by 29
Odutayo, A., da Costa, B. R., Pereira, T. V., Garg, V., Iskander, S., Roble, F., Lalji, R., Hincapié, C. A., Akingbade, A., Rodrigues, M., et al. (2021) · Journal of the American Heart Association · doi:10.1161/jaha.120.019918 · Cited by 29
Skin hyperpigmentation after sclerotherapy with polidocanol: A systematic review
Bossart, S., Daneluzzi, C., Cazzaniga, S., Ramelet, A., Uthoff, H., Jafari, S. M. S., Baumgartner, M., Hunger, R. E., Heidemeyer, K., & Willenberg, T. (2022) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.18639 · Cited by 28
Treatments for Burning Mouth Syndrome: A Network Meta-analysis
Alvarenga‐Brant, R., Costa, F. O., Pereira, G. H. M., Lima, R. P. E., Belém, F. V., Lai, H., Ge, L., Gómez, R. S., & Martins, C. C. (2022) · Journal of Dental Research · doi:10.1177/00220345221130025 · Cited by 28
“We rated down the certainty of evidence if there were problems due to the risk of bias, inconsistency, indirectness, publication bias, intransitivity, imprecision, and incoherence (Puhan et al. 2014; Bonner et al. 2018; Brignardello-Petersen, Bonner, et al. 2018).”
Efficacy of small-incision lenticule extraction surgery in high astigmatism: A meta-analysis
Cui, G., Yu, D., Yang, S., Chen, D., & Li, Y. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2022.1100241 · Cited by 28
“The certainty of the body of evidence was assessed as high, moderate, low, or very low, for the primary and secondary outcomes using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach (21).”
MethodZhang, L., Wang, F., Tashiro, S., & Liu, P. J. (2024) · Advances in Nutrition · doi:10.1016/j.advnut.2024.100330 · Cited by 36
Pitre, T., Lupas, D., Mah, J. C., Stanbrook, M. B., Blazer, A. J., Zeraatkar, D., & Ho, T. N. (2025) · COPD Journal of Chronic Obstructive Pulmonary Disease · doi:10.1080/15412555.2025.2449889 · Cited by 31
Linde, K., Rücker, G., Schneider, A., & Kriston, L. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.10.010 · Cited by 27
Kang, P., Park, S.-K., Yoo, S., Hur, M. S., Kim, W. H., Kim, J., & Bahk, J.-H. (2019) · Minerva Anestesiologica · doi:10.23736/s0375-9393.18.12813-6 · Cited by 29
Skandamis, A., Kani, C., Markantonis, S. L., & Souliotis, K. N. (2019) · Journal of Drug Assessment · doi:10.1080/21556660.2019.1597726 · Cited by 30
A Markov chain approach for ranking treatments in network meta‐analysis
Chaimani, A., Porcher, R., Sbidian, É., & Mavridis, D. (2020) · Statistics in Medicine · doi:10.1002/sim.8784 · Cited by 27
McGirr, A., Devoe, D. J. A., Raedler, A., Debert, C. T., Ismail, Z., & Berlim, M. T. (2020) · The Canadian Journal of Psychiatry · doi:10.1177/0706743720982432 · Cited by 29
“Transitivity was assessed by a global test for inconsistency23 and inconsistency plots assuming loop-specific heterogeneity to determine whether inconsistency existed in each NMA.24-27 Inconsistency was potentially mitigated a priori with the use of a strict inclusion criteria during the screening phases.”
MethodInfluentialSingh, N. P., Makkar, J. K., Cappellani, R. B., Sinha, A. C., Lakshminarasimhachar, A., & Singh, P. M. (2020) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-020-01792-4 · Cited by 28
“Risk of bias was assessed based upon the criteria recommended by the Cochrane Collaboration (ROB2).(13) This tool evaluates the ROB according to five domains: bias arising from the randomization process, bias due to deviation from intended interventions, bias due to missing data outcomes, bias in measurement of outcom…”
MethodMethods of Weaning From Mechanical Ventilation in Adult: A Network Meta-Analysis
Jhou, H., Chen, P., Ou‐Yang, L., Lin, C. S., Tang, S.-E., & Lee, C. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.752984 · Cited by 28
Pharmacological treatments in panic disorder in adults: a network meta-analysis
Guaiana, G., Meader, N., Barbui, C., Davies, S., Furukawa, T. A., Imai, H., Dias, S., Caldwell, D. M., Koesters, M., Tajika, A., et al. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012729.pub3 · Cited by 35
Network Meta-analysis: Users’ Guide for Surgeons: Part II - Certainty
Chaudhry, H., Foote, C. J., Guyatt, G. H., Thabane, L., Furukawa, T. A., Petrisor, B. A., & Bhandari, M. (2015) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-015-4287-9 · Cited by 28
Self-citationPike, E., Hamidi, V., Sæterdal, I., Odgaard‐Jensen, J., & Klemp, M. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014880 · Cited by 28
“The quality of the direct evidence, indirect evidence and the combined evidence from the NMAs was evaluated using the GRADE approach for network meta-analyses.(33) We used the Guideline Development Tool,(34) while evaluating the quality of the direct evidence.”
MethodMcLeod, S. L., Brignardello‐Petersen, R., Worster, A., You, J. J., Iansavitchene, A., Guyatt, G. H., & Cheskes, S. (2017) · Resuscitation · doi:10.1016/j.resuscitation.2017.10.012 · Cited by 28
Self-citationHuang, Y., Xie, W.-J., Fan, H., & Du, J. (2019) · Frontiers in Oncology · doi:10.3389/fonc.2019.00972 · Cited by 29
“TheGrading of Recommendations Assessment, Development, and Evaluation system (GRADE) approach was used to rate the quality of evidence (28).”
Using Systematic Reviews in Guideline Development
Schünemann, H. J. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch22 · Cited by 29
The development of network meta-analysis
Lee, A. (2022) · Journal of the Royal Society of Medicine · doi:10.1177/01410768221113196 · Cited by 27
“Also in 2011, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) published reports on the interpretation and conduct of NMA.13,14 Ades53 observed that this ‘seems to represent the first position statement from an academic body on these methods’.”
MethodInfluentialWei, Q., Wang, H., Zhao, J., Luo, Z., Wang, C., Zhu, C., Su, N., & Zhang, S. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1237234 · Cited by 29
“We evaluated the quality of the results in this study by applying the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach specifically designed for network meta-analysis (Puhan et al., 2014; Brignardello-Petersen et al., 2018; Brignardello-Petersen et al., 2019).”
MethodFacciorusso, A., Crinò, S. F., Fugazza, A., Carrara, S., Spadaccini, M., Colombo, M., Ramai, D., Mangiavillano, B., Chandan, S., Gkolfakis, P., et al. (2023) · Endoscopy · doi:10.1055/a-2156-0063 · Cited by 27
“Briefly, evidence from RCTs started at high quality, and was rated down for presence of any of the following factors – risk of bias in the literature, inconsistency, indirectness, imprecision, and publication bias [22].”
BackgroundMeta-analysis comparing chewing gum versus standard postoperative care after colorectal resection
Song, G.-M., Deng, Y.-H., Jin, Y., Zhou, J., & Tian, X. (2016) · Oncotarget · doi:10.18632/oncotarget.11735 · Cited by 27
“In this method, the evidence from RCTs was firstly established to has high quality and five down-grading factors including risk of bias, imprecision, indirectness, inconsistency, and publication bias can reduce the level to moderate, low and very low [57, 58].”
MethodNagasaka, K., Harigai, M., Hagino, N., Hara, A., Horita, T., Hayashi, T., Itabashi, M., Ito, S., Katsumata, Y., Kawashima, S., et al. (2018) · Modern Rheumatology · doi:10.1080/14397595.2018.1500111 · Cited by 27
“The GRADE working group reported NMA as a tool for comparing efficacy of multiple treatments [41].”
MethodKim, E. H., Park, S. W., Nam, E., Lee, J. G., & Park, C. H. (2018) · Surgical Endoscopy · doi:10.1007/s00464-018-6409-4 · Cited by 27
Where is the strength of evidence? A review of infection prevention and control guidelines
MITCHELL, B. G., Fasugba, O., & Russo, P. L. (2020) · Journal of Hospital Infection · doi:10.1016/j.jhin.2020.01.008 · Cited by 27
Interventions for great saphenous vein reflux: network meta-analysis of randomized clinical trials
Siribumrungwong, B., Wilasrusmee, C., Orrapin, S., Srikuea, K., Benyakorn, T., McKay, G. J., Attia, J., Rerkasem, K., & Thakkinstian, A. (2020) · British journal of surgery · doi:10.1093/bjs/znaa101 · Cited by 30
Fong, K. M., Au, S. Y., & Ng, G. W. Y. (2021) · Scientific Reports · doi:10.1038/s41598-021-95386-9 · Cited by 29
Azadeh, H., Alizadeh‐Navaei, R., Rezaiemanesh, A., & Rajabinejad, M. (2022) · Inflammopharmacology · doi:10.1007/s10787-022-00933-z · Cited by 29
“We used the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) model (via GRADpro/ GTD online software) to access the quality of the evidence (Puhan et al. 2014).”
MethodKim, W. H., Hur, M. S., Park, S.-K., Jung, D.-E., Kang, P., Yoo, S., & Bahk, J.-H. (2018) · Anaesthesia · doi:10.1111/anae.14227 · Cited by 27
Filippini, G., Clerico, M., Beiki, O., Mattoscio, M., Piazza, F., Giovane, C. D., Fredrikson, S., Tramacere, I., & Scalfari, A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012200 · Cited by 27
Lektip, C., Chaovalit, S., Wattanapisit, A., Lapmanee, S., Nawarat, J., & Yaemrattanakul, W. (2023) · PeerJ · doi:10.7717/peerj.15699 · Cited by 26
Magaña-Ramírez, M., Gallardo‐Gómez, D., Álvarez‐Barbosa, F., & Corral-Pernía, J. A. (2024) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2024.02.001 · Cited by 26
Sinopoulou, V., Nigam, G. B., Gordon, M. A., Ganeshan, M., Tokonyai, M. R., Dolwani, S., Iacucci, M., Rutter, M. D., Subramanian, V., Wilson, A., et al. (2024) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2024.11.008 · Cited by 26
Sridharan, K., & Sivaramakrishnan, G. (2018) · Expert Review of Clinical Pharmacology · doi:10.1080/17512433.2018.1530108 · Cited by 26
“Grading of the evidence for key comparisons was carried out using Grades of Recommendation, Assessment, Development, and Evaluation working group approach [18].”
MethodHazlewood, G. S., Whittle, S., Kamso, M. M., Akl, E. A., Wells, G. A., Tugwell, P., Thomas, M., Lee, C., Ejaredar, M., Choudhary, D., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013562 · Cited by 27
Shih, M., & Tu, Y. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1480 · Cited by 27
“In particular, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group, dedicated to a structured assessment of evidence quality, has incorporated the direct-indirect evidence inconsistency approach as the tool of choice to assess statistical incoherence within the network.(10,11)…”
BackgroundMethodBalneotherapy for Fibromyalgia Syndrome: A Systematic Review and Meta-Analysis
Cao, C.-F., Ma, K., Qian-lu, L., Luan, F.-J., Wang, Q., Zhang, M., Viswanath, O., Myrcik, D., Varrassi, G., & Wang, H. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10071493 · Cited by 27
“The strength of evidence was assessed using the online version of GRADEpro GDT software, identified as high, moderate, low, and very low [21].”
MethodDubravcsik, Z., Hritz, I., Keczer, B., Novák, P., Lovasz, B. D., & Madácsy, L. (2021) · Pancreatology · doi:10.1016/j.pan.2021.04.006 · Cited by 26
Godolphin, P. J., Fisher, D. J., Berry, L. R., Derde, L. P. G., Diaz, J. V., Gordon, A. C., Lorenzi, E., Marshall, J. C., Murthy, S., Shankar‐Hari, M., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0270668 · Cited by 27
Wu, X., Shi, J., Buti, J., Lai, H., & Tonetti, M. S. (2023) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13771 · Cited by 26
Zafar, M. I., Mills, K., Baird, C. D., Jiang, H., & Li, H. (2023) · Drugs · doi:10.1007/s40265-023-01853-0 · Cited by 28
“of the meta-analyses and network meta-analyses are presented as GRADE tables [55] using the template for continuous outcomes given in Yepes-Nuñez 2019 [56].”
MethodHussain, N., Brull, R., Vannabouathong, C., Robinson, C., Zhou, S., D'Souza, R. S., Sawyer, T. R., Terkawi, A. S., & Abdallah, F. W. (2023) · Anesthesiology · doi:10.1097/aln.0000000000004667 · Cited by 26
A threshold analysis assessed the credibility of conclusions from network meta-analysis
Caldwell, D. M., Ades, A. E., Dias, S., Watkins, S., Li, T., Taske, N., Naidoo, B., & Welton, N. J. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.07.003 · Cited by 26
“Adapted from [8].”
InfluentialSelf-citationSiemieniuk, R. A. C., Foroutan, F., Mirza, R. D., Ming, J. M., Alexander, P. E., Agarwal, A., Lesi, O. A., Merglen, A., Chang, Y., Zhang, Y., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-019022 · Cited by 26
“For the outcomes specific to HBV, we used the GRADE approach for rating certainty of network effect estimates obtained from a network meta-analysis.(31) In brief, we rated the certainty of evidence for direct comparisons using the standard GRADE approach.”
BackgroundMethodSelf-citationLiao, X., Qiu, S., Bao, Y., Wang, W., Yang, L., & Wei, Q. (2018) · World Journal of Urology · doi:10.1007/s00345-018-2583-1 · Cited by 26
“The quality of evidence on all comparisons was evaluated by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system [23].”
MethodLuo, M., Zhao, X., Deng, M., Hu, Y., Yang, X., Mei, Z., Meng, L., & Wang, Y. (2022) · Journal of Neurosurgical Anesthesiology · doi:10.1097/ana.0000000000000868 · Cited by 25
Lee, K.-H., Wang, J.-C., Chang, C.-Y., Teng, W.-N., Wang, H., Lee, C.-C., Tsai, S., Chen, C., Wu, P., & Chen, W.-M. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2025.04.019 · Cited by 25
Srisurapanont, K., Samakarn, Y., Kamklong, B., Siratrairat, P., Bumiputra, A., Jaikwang, M., & Srisurapanont, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246172 · Cited by 25
Effects of pharmacist interventions on heart failure outcomes: A systematic review and meta‐analysis
Arunmanakul, P., Kengkla, K., Chaiyasothi, T., Phrommintikul, A., Ruengorn, C., Permsuwan, U., Thakkinstian, A., Page, R. L., II, Munger, M. A., Nathisuwan, S., et al. (2021) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.1442 · Cited by 28
“The quality rating of RCTs started with high quality, which was then downgraded based on risk of bias, inconsistency (heterogeneity), indirectness, imprecision, and publication bias for direct estimates.(24,25)”
BackgroundHo, L. T. F., Zhong, C. C., Wong, C. H. L., Wu, J. C., Chan, K. K., Wu, X., Leung, T. H., & Chung, V. C. (2021) · Chinese Medicine · doi:10.1186/s13020-021-00556-6 · Cited by 26
“Quality of evidence was assessed for pairwise meta-analyses and NMAs using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach [23, 24].”
Method
Metadata
- Authors
- Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group
- Affiliations
- University of Zurich (CH)
- Mayo Clinic (US)
- Johns Hopkins University (US)
- University of Alabama (US)
- Maastricht University (NL)
- McMaster University (CA)
- Corresponding author
- Puhan, M. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2014-09-24
- Volume
- 349
- Issue
- sep24 5
- Pages
- g5630-g5630
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155798268
- Cited by
- 1,812
- References
- 39
- Citations by year
- 2026: 160 · 2025: 177 · 2024: 216 · 2023: 175 · 2022: 205 · 2021: 276 · 2020: 201 · 2019: 128 · 2018: 96 · 2017: 72 · 2016: 63 · 2015: 36 · 2014: 4
- FWCI
- 57.15
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-06-17)
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, GRADE, treatment effect estimates, comparative effectiveness
- MeSH terms
- Humans, Quality Assurance, Health Care, Meta-Analysis as Topic, Practice Guidelines as Topic, Evidence-Based Medicine, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review