The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
Open accessAbstract
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [45].”
From The Effect of Different Psychological Interventions on Athletes' Competitive Anxiety: A Systematic Review and Bayesian Network Meta-Analysis. (2026) · cites this paper as [45]
In the citing paper
“Therefore, we downgraded the quality of evidence for the comparison between SGLT2i and finerenone, SGLT2i and GLP-1 RA.”
In the citing paper
“Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head.”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
In the citing paper
“Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Items 201–300 of 1,001
Management strategies for pancreatic pseudocysts
Gurusamy, K. S., Pallari, E., Hawkins, N., Pereira, S. P., & Davidson, B. R. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011392.pub2 · Cited by 71
Aziz, M., Fatima, R., Dong, C., Lee‐Smith, W., & Nawras, A. T. (2020) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15070 · Cited by 71
D’Antonio, F., Berghella, V., Mascio, D. D., Saccone, G., Sileo, F. G., Flacco, M. E., Odibo, A. O., Liberati, M., Manzoli, L., & Khalil, A. A. (2021) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2021.04.023 · Cited by 76
Kanie, T., Mizuno, A., Takaoka, Y., Suzuki, T., Yoneoka, D., Nishikawa, Y., Tam, W., Morze, J., Rynkiewicz, A., Xin, Y., et al. (2021) · UCL Discovery (University College London) · Cited by 71
Singh, J. A., Hossain, A., Tanjong-Ghogomu, E., Mudano, A., Tugwell, P., & Wells, G. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012437 · Cited by 73
Self-citationMaintenance immunosuppression for adults undergoing liver transplantation: a network meta-analysis
Rodríguez‐Perálvarez, M., Misas, M. G., Thorburn, D., Davidson, B. R., Tsochatzis, E. A., & Gurusamy, K. S. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011639.pub2 · Cited by 70
Khan, M. S., Khan, F., Fonarow, G. C., Sreenivasan, J., Greene, S. J., Khan, S. U., Usman, M., Vaduganathan, M., Fudim, M., Anker, S. D., et al. (2021) · European Journal of Heart Failure · doi:10.1002/ejhf.2278 · Cited by 69
Tocolytics for delaying preterm birth: a network meta-analysis (0924)
Wilson, A., Morton, V. H., Marson, E. J., Markland, A. D., Larkai, E., Papadopoulou, A., Coomarasamy, A., Tobı́as, A., Chou, D., Oladapo, O. T., et al. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd014978.pub2 · Cited by 74
Jalalzadeh, H., Groenen, H., Buis, D. R., Dreissen, Y. E. M., Goosen, J. H. M., IJpma, F. F. A., van der Laan, M. J., Schaad, R. R., Segers, P., van der Zwet, W. C., et al. (2022) · The Lancet Microbe · doi:10.1016/s2666-5247(22)00187-2 · Cited by 78
Antonacci, D., Caponio, V. C. A., Troiano, G., Pompeo, M. G., Gianfreda, F., & Canullo, L. (2022) · Journal of Prosthodontic Research · doi:10.2186/jpr.jpr_d_22_00107 · Cited by 74
Brignardello‐Petersen, R., TOMLINSON, G. A., Flórez, I. D., Rind, D. M., Chu, D. K., Morgan, R. L., Mustafa, R. A., Schünemann, H. J., & Guyatt, G. H. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.06.010 · Cited by 72
Self-citationKanters, S., Socias, M. E., Paton, N. I., Vitória, M., Doherty, M. C., Ayers, D., Popoff, E., Chan, K., Cooper, D. A., Wiens, M. O., et al. (2017) · The Lancet HIV · doi:10.1016/s2352-3018(17)30109-1 · Cited by 69
S3 Guideline: Chronic Tinnitus
Mazurek, B., Hesse, G., Sattel, H. C., Kratzsch, V., Lahmann, C., & Dobel, C. (2022) · HNO · doi:10.1007/s00106-022-01207-4 · Cited by 67
The use of GRADE approach in systematic reviews of animal studies
Wei, D., Tang, K., Wang, Q., Estill, J. A. M., Yao, L., Wang, X., Chen, Y., & Yang, K. (2016) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12198 · Cited by 71
“Currently, consensus and guidance exist for the use of GRADE in SRs of interventions (4–9), diagnostic accuracy (10), prognosis studies (11, 12), and network meta-analyses (13, 14).”
BackgroundNguyen, N. H., Fuméry, M., Dulai, P. S., Prokop, L. J., Sandborn, W. J., Murad, M. H., & Singh, S. (2018) · The Lancet. Gastroenterology & hepatology · doi:10.1016/s2468-1253(18)30231-0 · Cited by 71
Self-citationBelland, B. R., Walker, A. E., & Kim, N. J. (2017) · Review of Educational Research · doi:10.3102/0034654317723009 · Cited by 90
“When taking a frequentist approach to network meta-analysis, all included studies need to contain a treatment and a control condition (Puhan et al., 2014).”
MethodRokkas, T., Gisbert, J. P., Gasbarrini, A., Hold, G. L., Tilg, H., Malfertheiner, P., Mégraud, F., & O'Morain, C. A. (2019) · United European Gastroenterology Journal · doi:10.1177/2050640619854587 · Cited by 69
“This NWM was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension statement for interventions,(23) whereas the rating of the quality of RCTs was achieved by using the Grading of Recommendations Assessment, Development and Evaluation Working Group modality.(24) Further…”
MethodKhan, S. U., Talluri, S., Riaz, H., Rahman, H., Nasir, F., Bin Riaz, I., Sattur, S., Ahmed, H., Kaluski, E., & Krasuski, R. (2018) · European Journal of Preventive Cardiology · doi:10.1177/2047487318766612 · Cited by 61
Hersi, M., Traversy, G., Thombs, B. D., Beck, A., Skidmore, B., Groulx, S., Lang, E., Reynolds, D. L., Wilson, B. J., Bernstein, S. L., et al. (2019) · Systematic Reviews · doi:10.1186/s13643-018-0928-x · Cited by 66
Wang, Y., Ge, L., Ye, Z., Siemieniuk, R. A. C., Blaser, A. R., Wang, X., Perner, A., Møller, M. H., Alhazzani, W. A., Cook, D., et al. (2020) · Intensive Care Medicine · doi:10.1007/s00134-020-06209-w · Cited by 69
Self-citationTan‐Lim, C. S. C., Esteban‐Ipac, N. A. R., Recto, M. S. T., Castor, M. A. R., Casis‐Hao, R. J., & Nano, A. L. M. (2021) · Pediatric Allergy and Immunology · doi:10.1111/pai.13514 · Cited by 65
Treating child and adolescent anxiety effectively: Overview of systematic reviews
BENNETT, K. J., Manassis, K., Duda, S. J., Bagnell, A. L., Bernstein, G. A., Garland, E., Miller, L. D., Newton, A. S., Thabane, L., & Wilansky, P. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.09.006 · Cited by 90
“Comparative effectiveness findings based on a network meta-analysis are available, but questions concerning validity and the interpretation of network meta-analysis findings require further investigation (Mills et al., 2013; Puhan et al., 2014).”
BackgroundChong, H. Y., Lai, N. M., Apisarnthanarak, A., & Chaiyakunapruk, N. (2017) · Clinical Infectious Diseases · doi:10.1093/cid/cix019 · Cited by 66
Kongpakwattana, K., Sawangjit, R., Tawankanjanachot, I., Bell, J. S., Hilmer, S. N., & Chaiyakunapruk, N. (2018) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.13604 · Cited by 67
Tarp, S., Furst, D. E., Boers, M., Luta, G., Bliddal, H., Tarp, U., Asmussen, K. H., Brock, B., Døssing, A., Jørgensen, T. S., et al. (2016) · Lara D. Veeken · doi:10.1093/rheumatology/kew442 · Cited by 63
Description of network meta-analysis geometry: A metrics design study
Tonin, F. S., Borba, H. H. L., Mendes, A. M., Wiens, A., Fernández-Llimós, F., & Pontarolo, R. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212650 · Cited by 65
Park, C. H., Jung, D. H., Kim, D. H., Lim, C., Moon, H. S., Park, J. H., Jung, H., Hong, S. J., Choi, S. C., & Lee, O. Y. (2019) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2019.05.046 · Cited by 62
Uterotonic agents for first-line treatment of postpartum haemorrhage: a network meta-analysis
Parry‐Smith, W., Papadopoulou, A., Thomas, E., Tobı́as, A., Price, M. J., Meher, S., Alfirević, Ž., Weeks, A. D., Hofmeyr, G. J., Gülmezoglu, A. M., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012754.pub2 · Cited by 66
Kim, M. S., Seong, D., Li, H., Chung, S. K., Park, Y., Lee, M., Lee, S. W., Yon, D. K., Kim, J. H., Lee, K. H., et al. (2022) · Reviews in Medical Virology · doi:10.1002/rmv.2336 · Cited by 65
Dost, B., CASSAI, A. D., Balzani, E., Tulgar, S., & Ahişkalıoğlu, A. (2022) · BMC Anesthesiology · doi:10.1186/s12871-022-01952-7 · Cited by 61
Comparative efficacy of interventions on nonalcoholic fatty liver disease (NAFLD)
Sawangjit, R., Chongmelaxme, B., Phisalprapa, P., Saokaew, S., Thakkinstian, A., Kowdley, K. V., & Chaiyakunapruk, N. (2016) · Medicine · doi:10.1097/md.0000000000004529 · Cited by 61
Therapies for bruxism: a systematic review and network meta-analysis (protocol)
Mesko, M. E., Hutton, B., Skupien, J. A., Sarkis‐Onofre, R., Moher, D., & Pereira‐Cenci, T. (2017) · Systematic Reviews · doi:10.1186/s13643-016-0397-z · Cited by 68
“This approach is based on four steps considering direct and indirect treatment estimates for each comparison of the evidence network, rating the quality of each direct and indirect effect estimate, rating the NMA estimate for each comparison of the evidence network and quality of each NMA effect estimate [32].”
MethodKomolafe, O., Roberts, D. A., Freeman, S. C., Wilson, P., Sutton, A. J., Cooper, N. J., Pavlov, C. S., Milne, E., Hawkins, N., Cowlin, M., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013125.pub2 · Cited by 64
Snedecor, S. J., Radford, M., Kratochvil, D., Grove, R. A., & Punekar, Y. S. (2019) · BMC Infectious Diseases · doi:10.1186/s12879-019-3975-6 · Cited by 61
Use of peers to improve adherence to antiretroviral therapy: a global network meta‐analysis
Kanters, S., Park, J., Chan, K., Ford, N. P., Forrest, J. I., Thorlund, K., Nachega, J. B., & Mills, E. J. (2016) · Journal of the International AIDS Society · doi:10.7448/ias.19.1.21141 · Cited by 61
“To assess the overall strength of evidence, we employed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system for NMA (Supplementary Tables 3 6) [11].”
MethodPlaner, D., Yanko, S., Matok, I., Paltiel, O., Zmiro, R., Rotshild, V., Amir, O., Elbaz‐Greener, G., & Raccah, B. H. (2023) · Canadian Medical Association Journal · doi:10.1503/cmaj.220960 · Cited by 65
Crocker, T. F., Ensor, J., Lam, N., Jordão, M., Bajpai, R. C., Bond, M., Förster, A., Riley, R. D., Andre, D., Brundle, C., et al. (2024) · BMJ · doi:10.1136/bmj-2023-077764 · Cited by 77
“16-19 21 We used plain language terms “probably” and “may” to indicate moderate and low certainty respectively and qualified the size and direction of point estimates (see appendix 5).”
MethodWyles, C. C., Jiménez-Almonte, J. H., Murad, M. H., Norambuena, G. A., Cabanela, M. E., Sierra, R. J., & Trousdale, R. T. (2014) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-014-4065-0 · Cited by 58
“The RRs of revision surgery with lower level and upper level 95% CrIs were also calculated for each pairwise comparison (Table 4).”
BackgroundMethodInfluentialSelf-citationWilliams, T., McCaul, M., Schwarzer, G., Cipriani, A., Stein, D. J., & Ipser, J. (2020) · Acta Neuropsychiatrica · doi:10.1017/neu.2020.6 · Cited by 59
“The approach by Puhan et al. (2014) was also used to assess indirect comparisons.”
MethodBalasubramanian, P., Abia‐Trujillo, D., Barrios-Ruiz, A., Garza-Salas, A., Koratala, A., Chandra, N. C., Lee‐Mateus, A. Y., Gonzalo Labarca, T., & Fernández-Bussy, S. (2024) · European Respiratory Review · doi:10.1183/16000617.0046-2024 · Cited by 61
Li, X., Zhang, L., Gu, S., Sun, J., Qin, Z., Yue, J., Zhong, Y., Ding, N., & Gao, R. (2018) · Medicine · doi:10.1097/md.0000000000012819 · Cited by 62
“After all the outcomes were evaluated, a summary of findings table was created using the GRADE system,([29]) following the 4step approach to rate the quality of evidence in each of the direct and mixed estimates based on methods developed by the GRADE working group.([30,31])”
MethodKanie, T., Mizuno, A., Takaoka, Y., Suzuki, T., Yoneoka, D., Nishikawa, Y., Tam, W. W. S., Morze, J., Rynkiewicz, A., Xin, Y., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013650.pub2 · Cited by 80
Gulea, C., Zakeri, R., Alderman, V., Morgan, A., Ross, J., & Quint, J. K. (2021) · Respiratory Research · doi:10.1186/s12931-021-01661-8 · Cited by 59
“Additionally, we assessed the certainty of the evidence using the Grading of Recommendations Assessment Development and Evaluation (GRADE) framework [16].”
MethodMeulenkamp, B., Woolnough, T., Cheng, W., Shorr, R., Stacey, D., Richards, M., Gupta, A. A., Fergusson, D. A., & Graham, I. D. (2021) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001861 · Cited by 59
“The overall quality of the evidence was determined and ranked per the Grades of Recommendation, Assessment, Development, and Evaluation approach for network meta-analyses [25, 71].”
MethodForget, P., Borovac, J. A., Thackeray, E. M., & Pace, N. L. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003006.pub4 · Cited by 60
“'Summary of findings' table and GRADE Methods for evaluating the quality of evidence of a NMA have been proposed by Puhan and colleagues (Puhan 2014), and Salanti and colleagues (Salanti 2014).”
BackgroundYao, L., Sadeghirad, B., Li, M., Li, J., Wang, Q., Crandon, H. N., Martin, G., Morgan, R., Florez, I. D., Hunskaar, B. S., et al. (2023) · BMJ · doi:10.1136/bmj-2023-076226 · Cited by 57
“45-47 45-47 Our assessment of transitivity relied on two fundamental issues: (1) eligible trials are jointly randomisable, and (2) potential effect modifiers are equally distributed between each treatment comparison in the network.”
BackgroundSelf-citationSeow, W., Dudi-Venkata, N. N., Bedrikovetski, S., Kroon, H. M., & Sammour, T. (2022) · Techniques in Coloproctology · doi:10.1007/s10151-022-02739-1 · Cited by 59
“Additional details on GRADE analysis methods are outlined are available online (Appendix S3, supporting information).”
MethodInfluentialIncisional Wound Irrigation for the Prevention of Surgical Site Infection
Groenen, H., Bontekoning, N., Jalalzadeh, H., Buis, D. R., Dreissen, Y. E. M., Goosen, J. H. M., Graveland, H., Griekspoor, M., IJpma, F. F. A., van der Laan, M. J., et al. (2024) · JAMA Surgery · doi:10.1001/jamasurg.2024.0775 · Cited by 57
Bazerbachi, F., Haffar, S., Sawas, T., Vargas, E. J., Kaur, R. J., Wang, Z., Prokop, L. J., Murad, M. H., & Dayyeh, B. K. A. (2018) · Obesity Surgery · doi:10.1007/s11695-018-3227-7 · Cited by 57
Self-citationBradley, A., & Meer, R. V. D. (2019) · Scientific Reports · doi:10.1038/s41598-019-40951-6 · Cited by 56
Wang, X., Martin, G., Sadeghirad, B., Chang, Y., Flórez, I. D., Couban, R., Mehrabi, F., Crandon, H. N., Esfahani, M. A., Sivananthan, L., et al. (2025) · BMJ · doi:10.1136/bmj-2024-079971 · Cited by 61
“53 55 We assessed for intransitivity considering two fundamental issues: 1) exploring the similarity of patient populations in our networks and confirming with our clinical experts that patients across trials were eligible to receive any of interventions considered in the network; and 2)…”
Self-citationWoolnough, T., Axelrod, D. E., Bozzo, A., Koziarz, A., Koziarz, F., Oitment, C. T. A., Gyemi, L. A., Gormley, J., Gouveia, K., & Johal, H. S. (2020) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001524 · Cited by 59
“The quality of each included study was evaluated in duplicate using the Cochrane Risk of Bias tool, and the Cochrane Confidence in Network Meta-Analysis (CINeMA) tool was used for the risk of bias assessment specific to NMAs [18, 46].”
MethodMiroshnychenko, A., Ibrahim, S., Azab, M., Roldán, Y., Martinez, J. P. D., Tamilselvan, D., He, L., Little, J. W., Urquhart, O., Tampi, M. P., et al. (2023) · Journal of Dental Research · doi:10.1177/00220345221139230 · Cited by 57
“Appendix Figures 1 to 21.6 and Appendix Tables 12 to 28 include network plots and forest plots of pairwise meta-analyses for all outcomes, as well as detailed relative and absolute effect estimates from the NMAs and the certainty of evidence for all comparisons and outcomes.”
MethodInfluentialSelf-citationFilippini, G., Giovane, C. D., Clerico, M., Beiki, O., Mattoscio, M., Piazza, F., Fredrikson, S., Tramacere, I., Scalfari, A., & Salanti, G. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012200.pub2 · Cited by 58
Song, G.-M., Liu, X., Bian, W., Wu, J., Deng, Y.-H., Zhang, H., & Tian, X. (2017) · Oncotarget · doi:10.18632/oncotarget.15580 · Cited by 55
Jarde, A., Lutsiv, O., Park, C., Beyene, J., Dodd, J. M., Barrett, J. F. R., Shah, P. S., Cook, J., Saito, S., Biringer, A., et al. (2017) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.14624 · Cited by 55
Bessissow, T., Dulai, P. S., Restellini, S., Landry, T., Bisschops, R., Murad, M. H., & Singh, S. (2018) · Inflammatory Bowel Diseases · doi:10.1093/ibd/izy188 · Cited by 55
Self-citationAntiemetics in Children With Acute Gastroenteritis: A Meta-analysis
Niño-Serna, L. F., Acosta-Reyes, J. L., Veroniki, A. A., & Flórez, I. D. (2020) · PEDIATRICS · doi:10.1542/peds.2019-3260 · Cited by 63
Ye, Z., Wang, Y., Ge, L., Guyatt, G. H., Collister, D., Alhazzani, W. A., Bagshaw, S. M., Belley‐Côté, E. P., Fang, F., Hou, L., et al. (2021) · Critical Care Explorations · doi:10.1097/cce.0000000000000399 · Cited by 60
Self-citationCardiac Rehabilitation Programs for Chronic Heart Disease: A Bayesian Network Meta-analysis
Huang, R., Palmer, S. C., Cao, Y., Zhang, H., Sun, Y., Su, W., Liang, L., Wang, S., Wang, Y., Xu, Y., et al. (2020) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2020.02.072 · Cited by 56
Pharmacological interventions for primary biliary cholangitis
Saffioti, F., Gurusamy, K. S., Eusebi, L. H., Tsochatzis, E. A., Davidson, B. R., & Thorburn, D. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011648.pub2 · Cited by 54
Jakob, T., Tesfamariam, Y. M., Macherey‐Meyer, S., Kuhr, K., Adams, A., Monsef, I., Heidenreich, A., & Skoetz, N. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013020.pub2 · Cited by 54
Tegegne, T. K., Rawstorn, J. C., Nourse, R. A., Kibret, K. T., Ahmed, K. Y., & Maddison, R. (2022) · Open Heart · doi:10.1136/openhrt-2021-001949 · Cited by 58
Moncion, K., Rodrigues, L., Wiley, E., Noguchi, K. S., Negm, A. M., Richardson, J. A., MacDonald, M. J., Roig, M., & Tang, A. (2024) · British Journal of Sports Medicine · doi:10.1136/bjsports-2023-107956 · Cited by 52
Costa, G. D. M., Zanatta, F. B., Ziegelmann, P. K., Barros, A. J. S., & Mello, C. F. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2020.07.046 · Cited by 57
Endoscopic Treatment of Large Bile Duct Stones: A Systematic Review and Network Meta-Analysis
Facciorusso, A., Gkolfakis, P., Ramai, D., Tziatzios, G., Lester, J., Crinò, S. F., Frazzoni, L., Papanikolaou, I. S., Arvanitakis, M., Bléro, D., et al. (2021) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2021.10.013 · Cited by 57
Pulikkotil, S. J., Nagendrababu, V., Veettil, S. K., Jinatongthai, P., & Setzer, F. C. (2018) · International Endodontic Journal · doi:10.1111/iej.12912 · Cited by 56
Tian, G., Yang, S., Yuan, J.-Q., Threapleton, D. E., Zhao, Q., Chen, F., Cao, H., Jiang, T., & Li, L. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-021269 · Cited by 52
Yasuda, H., Okano, H., Mayumi, T., Narita, C., Onodera, Y., Nakane, M., & Shime, N. (2021) · Critical Care · doi:10.1186/s13054-021-03550-4 · Cited by 52
“This study utilized a four-step approach for assessing the certainty of the NMA estimate developed by the GRADE Working Group [51], whereas the study by Zhou et al. did not conduct a similar assessment.”
MethodInfluentialAlotaibi, F. F., Rocchietta, I., Buti, J., & D’Aiuto, F. (2023) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13850 · Cited by 54
“Certainty in the evidence (CiE) was assessed at the outcome level using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach (Puhan et al., 2014).”
MethodNetwork Meta-analysis: Users’ Guide for Surgeons: Part I - Credibility
Foote, C. J., Chaudhry, H., Bhandari, M., Thabane, L., Furukawa, T. A., Petrisor, B. A., & Guyatt, G. H. (2015) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-015-4286-x · Cited by 51
Self-citationTäger, T., Atar, D., Agewall, S., Katus, H. A., Grundtvig, M., Cleland, J. G. F., Clark, A. L., Fröhlich, H., & Frankenstein, L. (2020) · Heart Failure Reviews · doi:10.1007/s10741-020-09954-8 · Cited by 53
Fountain, D. M., Bryant, A., Barone, D. G., Waqar, M., Hart, M. G., Bulbeck, H., Kernohan, A., Watts, C., & Jenkinson, M. D. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013630.pub2 · Cited by 56
“Cochrane Database of Systematic Reviews guidelines (Brignardello-Petersen 2018; Brignardello-Petersen 2019a; Brignardello-Petersen 2019b; Puhan 2014).”
MethodNickel, K., Halfpenny, N. J. A., Snedecor, S. J., & Punekar, Y. S. (2021) · BMC Infectious Diseases · doi:10.1186/s12879-021-05850-0 · Cited by 51
Yang, C., Yan, P., Wu, X., Zhang, W., Cui, H., Zhang, L., Xu, Z., Peng, S., Tang, M., Wang, Y., et al. (2024) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2024.101965 · Cited by 53
Dexamethasone versus betamethasone for preterm birth: a systematic review and network meta-analysis
Ciapponi, A., Klein, K., Colaci, D., Althabe, F., Belizán, J. M., Deegan, A., Veroniki, A. A., & Flórez, I. D. (2021) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2021.100312 · Cited by 52
Carron, M., Tamburini, E., Linassi, F., Pettenuzzo, T., Boscolo, A., & Navalesi, P. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.08.009 · Cited by 49
Yang, M. G., Wang, H.-T., Zhao, M., Meng, W., Ou, J.-Q., He, J., Zou, B., & Lei, P. (2015) · Medicine · doi:10.1097/md.0000000000001592 · Cited by 49
Park, C. H., Jung, Y. S., Nam, E., Eun, C. S., Park, D. I., & Han, D. S. (2016) · The American Journal of Gastroenterology · doi:10.1038/ajg.2016.287 · Cited by 49
Antibiotics for uncomplicated skin abscesses: systematic review and network meta-analysis
Wang, W., Chen, W., Liu, Y., Siemieniuk, R. A. C., Li, L., Martinez, J. P. D., Guyatt, G. H., & Sun, X. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-020991 · Cited by 54
“the quality of evidence in each of the direct, indirect and NMA estimates.(22) The rating of indirect estimates starts”
BackgroundSelf-citationMa, Y., Wang, C., Zhang, Q., Peng, X., Feng, Y., & Meng, X. (2018) · Pharmacological Research · doi:10.1016/j.phrs.2018.04.010 · Cited by 49
Russo, G. I., Cacciamani, G. E., Cocci, A., Kessler, T. M., Morgia, G., Şerefoğlu, E. C., Albersen, M., & Verze, P. (2019) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2018.12.011 · Cited by 49
In network meta-analysis, most of the information comes from indirect evidence: empirical study
Papakonstantinou, T., Nikolakopoulou, A., Egger, M., & Salanti, G. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.04.009 · Cited by 48
“In contrast, the GRADE system somewhat simplistically focuses on the most influential path of length 2, along with the direct evidence (28,29).”
BackgroundAssessing Confidence in the Results of Network Meta-Analysis (Cinema)Preprint
Nikolakopoulou, A., Higgins, J. P. T., Papakonstantinou, T., Chaimani, A., Giovane, C. D., Egger, M., & Salanti, G. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/597047 · Cited by 49
“Direct comparisons (number of studies) CCTA vs Exercise ECG (1) CCTA vs SPECT-MPI (2) CCTA vs Standard care (7) CMR vs Standard care (2) Exercise ECG vs Standard care (1) Exercise ECG vs Stress Echo (4) SPECT-MPI vs Standard care (2) Standard care vs Stress Echo (1)”
BackgroundMethodInfluentialMarcucci, M., Etxeandia‐Ikobaltzeta, I., Yang, S. S., Germini, F., Gupta, S., Agarwal, A., Ventresca, M., Tang, S., Morgano, G. P., Wang, M., et al. (2022) · BMJ · doi:10.1136/bmj-2021-066785 · Cited by 48
Self-citationZhou, S., Li, P., Lv, X., Lai, X., Liu, Z., Zhou, J., Liu, F., Tao, Y., Zhang, M., Yu, X., et al. (2023) · SLEEP · doi:10.1093/sleep/zsad177 · Cited by 48
Sensitivity of Treatment Recommendations to Bias in Network Meta-Analysis
Phillippo, D., Dias, S., Ades, A. E., Didelez, V., & Welton, N. J. (2017) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12341 · Cited by 50
“Recently, two methods to extend the GRADE framework to NMA have been proposed (Puhan et al. , 2014; Salanti et al. , 2014).”
Al‐Moraissi, E. A., Al-Tairi, N. H., Abotaleb, B. M., Al-Iryani, G. M., Halboub, E. S., & Al-Ak’hali, M. S. (2018) · Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.joms.2018.08.009 · Cited by 49
Chen, X., Chamoli, U., Lapkin, S., Castillo, J. V., & Diwan, A. D. (2019) · European Spine Journal · doi:10.1007/s00586-019-06142-7 · Cited by 50
Li, Z., Zhu, G., Li, C., Lai, H., Liu, X., & Zhang, L. (2021) · Nutrients · doi:10.3390/nu13124319 · Cited by 55
“In addition, we used the GRADE guideline to assess the certainty of the evidence for each outcome as high, moderate, low, or very low [22,23].”
MethodHu, Z., Ching, L. W., Hui-juan, L., Liang, Y., Zhiyu, W., Weiyang, H., Zhaoxiang, B., & Zhong, L. L. D. (2021) · Scientific Reports · doi:10.1038/s41598-021-88263-y · Cited by 50
Impact of center and endoscopist ERCP volume on ERCP outcomes: a systematic review and meta-analysis
de Campos, S. T., Papaefthymiou, A., Florou, T., Facciorusso, A., Arvanitakis, M., Deviere, J. M. E., & Gkolfakis, P. (2023) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2023.05.045 · Cited by 47
Lee, Y., Tahir, U., Tessier, L., Yang, K., Hassan, T., Dang, J. T., Kroh, M., & Hong, D. (2023) · Surgical Endoscopy · doi:10.1007/s00464-023-10151-5 · Cited by 52
Antiviral Medications for Treatment of Nonsevere Influenza
Gao, Y., Zhao, Y., Liu, M., Luo, S., Chen, Y., Chen, X., Zheng, Q., Xu, J., Shen, Y., Zhao, W., et al. (2025) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.7193 · Cited by 53
Self-citationNagendrababu, V., Pulikkotil, S. J., Jinatongthai, P., Veettil, S. K., Teerawattanapong, N., & Gutmann, J. L. (2019) · Journal of Endodontics · doi:10.1016/j.joen.2018.10.016 · Cited by 48
Ren, H., Liu, J., Liu, Y., Yu, C., Bao, G., & Kang, H. (2021) · Journal of Oral Rehabilitation · doi:10.1111/joor.13230 · Cited by 47
Xu, J.-J., Ren, H., Zhao, S.-W., Li, Q., Li, C., Bao, G., & Kang, H. (2023) · Head & Face Medicine · doi:10.1186/s13005-023-00369-y · Cited by 47
“Finally, to evaluate the evidence quality, the GRADE system was adopted according to the results of the above network meta-analysis [29].”
MethodComparative Effectiveness of Pharmacologic Interventions for Pulmonary Arterial Hypertension
Jain, S., Khera, R., Girotra, S. R., Badesch, D. B., Wang, Z., Murad, M. H., Blevins, A., Schmidt, G. A., Singh, S., & Gerke, A. K. (2016) · CHEST Journal · doi:10.1016/j.chest.2016.08.1461 · Cited by 47
“Galie(33)/2008 (ARIES 2) 50 (16) 51 (14) 81 68 65 35 65 36 355 (84) 343 (86) 2 44 52 2 3 37 57 3 None NA NA”
Self-citationEfficacy and Safety of Antibiotic Therapy in Early Cutaneous Lyme Borreliosis
Torbahn, G., Hofmann, H., Rücker, G., Bischoff, K., Freitag, M., Dersch, R. S., Fingerle, V., Motschall, E., Meerpohl, J. J., & Schmucker, C. M. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.3186 · Cited by 46
Lin, W., Zhong, M., Zhou, Q.-H., Zhang, Y.-R., Wang, H., Zhao, Z., Cheng, B., & Ling, C. (2019) · Cancer Management and Research · doi:10.2147/cmar.s195935 · Cited by 46
“Quality of evidence We assessed the quality of evidence of estimates derived from network meta-analysis using the GRADE methodology.(12,19) For direct comparisons in this system, RCTs start at high quality and may be downgraded to levels of moderate, low, and very low quality owing to heterogeneity, risk of bias, indi…”
Method
Metadata
- Authors
- Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group
- Affiliations
- University of Zurich (CH)
- Mayo Clinic (US)
- Johns Hopkins University (US)
- University of Alabama (US)
- Maastricht University (NL)
- McMaster University (CA)
- Corresponding author
- Puhan, M. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2014-09-24
- Volume
- 349
- Issue
- sep24 5
- Pages
- g5630-g5630
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155798268
- Cited by
- 1,812
- References
- 39
- Citations by year
- 2026: 160 · 2025: 177 · 2024: 216 · 2023: 175 · 2022: 205 · 2021: 276 · 2020: 201 · 2019: 128 · 2018: 96 · 2017: 72 · 2016: 63 · 2015: 36 · 2014: 4
- FWCI
- 57.15
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-06-17)
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, GRADE, treatment effect estimates, comparative effectiveness
- MeSH terms
- Humans, Quality Assurance, Health Care, Meta-Analysis as Topic, Practice Guidelines as Topic, Evidence-Based Medicine, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review