The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
Open accessAbstract
Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [45].”
From The Effect of Different Psychological Interventions on Athletes' Competitive Anxiety: A Systematic Review and Bayesian Network Meta-Analysis. (2026) · cites this paper as [45]
In the citing paper
“Therefore, we downgraded the quality of evidence for the comparison between SGLT2i and finerenone, SGLT2i and GLP-1 RA.”
In the citing paper
“Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head.”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
In the citing paper
“Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5”
From Drug treatments for covid-19: living systematic review and network meta-analysis (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320
Self-citationCombination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies
Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1044 · Cited by 2,098
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Cited by
Sorted by citations
Items 101–200 of 1,001
Welch, V. A., Ghogomu, E. A. T., Hossain, A., Awasthi, S., Bhutta, Z. A., Cumberbatch, C., Fletcher, R. H., McGowan, J., Krishnaratne, S., Kristjansson, E., et al. (2016) · The Lancet Global Health · doi:10.1016/s2214-109x(16)30242-x · Cited by 130
Use of network meta-analysis in clinical guidelines
Kanters, S., Ford, N., Druyts, E., Thorlund, K., Mills, E. J., & Bansback, N. (2016) · Bulletin of the World Health Organization · doi:10.2471/blt.16.174326 · Cited by 121
Kristensen, F. B., Husereau, D., Huić, M., Drummond, M., Berger, M. L., Bond, K., Augustovski, F., Booth, A., Bridges, J. F. P., Grimshaw, J., et al. (2018) · Value in Health · doi:10.1016/j.jval.2018.08.010 · Cited by 134
Yin, S., Xu, P., Wang, B., Lu, Y., Wu, Q., Zhou, M.-L., Wu, J., Cai, J., Sun, X., & Yuan, H. (2019) · BMJ · doi:10.1136/bmj.l2222 · Cited by 121
Interventions for necrotising pancreatitis
Gurusamy, K. S., Belgaumkar, A. P., Haswell, A., Pereira, S. P., & Davidson, B. R. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011383.pub2 · Cited by 118
Prophylaxis against covid-19: living systematic review and network meta-analysis
Bartoszko, J. J., Siemieniuk, R. A., Kum, E., Qasim, A., Zeraatkar, D., Martinez, J. P. D., Azab, M., Ibrahim, S., Izcovich, A., Soto, G. B., et al. (2021) · BMJ · doi:10.1136/bmj.n949 · Cited by 113
Self-citationHäggman‐Henrikson, B., Alstergren, P. J., Davidson, T., Högestätt, E. D., Östlund, P., Tranæus, S., Vitols, S., & List, T. (2017) · Journal of Oral Rehabilitation · doi:10.1111/joor.12539 · Cited by 128
“The SBU system uses different terminology compared to the GRADE working group replacing ‘high quality’ with ‘strong evidence’, ‘moderate quality’ with ‘moderately strong evidence’, ‘low quality’ with ‘limited evidence’ and ‘very low quality’ with ‘insufficient evidence’ (23).”
Speyer, H., Jakobsen, A. S., Westergaard, C., Nørgaard, H. C. B., Pisinger, C., Krogh, J., Hjorthøj, C., Nordentoft, M., Gluud, C., Correll, C. U., et al. (2019) · Psychotherapy and Psychosomatics · doi:10.1159/000502293 · Cited by 116
Methods to decrease blood loss during liver resection: a network meta-analysis
Moggia, E., Rouse, B., Simillis, C., Li, T., Vaughan, J., Davidson, B. R., & Gurusamy, K. S. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010683.pub3 · Cited by 112
Self-citationInterventions for central serous chorioretinopathy: a network meta-analysis
Salehi, M., Wenick, A. S., Law, H. A., Evans, J. R., & Gehlbach, P. L. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011841.pub2 · Cited by 141
Systematic review and network meta‐analysis: efficacy of drugs for functional dyspepsia
Ford, A. C., Moayyedi, P., Black, C. J., Yuan, Y., Veettil, S. K., Mahadeva, S., Kengkla, K., Chaiyakunapruk, N., & Lee, Y. Y. (2020) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.16072 · Cited by 119
Katsanos, K. H., Kitrou, P. M., Spiliοpoulos, S., Διαμαντόπουλος, Α., & Karnabatidis, D. (2016) · Journal of Endovascular Therapy · doi:10.1177/1526602816671740 · Cited by 112
Catalá-López, F., Macías Saint-Gerons, D., González-Bermejo, D., Rosano, G. M., Davis, B. R., Ridao, M., Zaragoza, A., Montero-Corominas, D., Tobías, A., de la Fuente-Honrubia, C., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1001971 · Cited by 109
“the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology [61,62] to evaluate the quality of evidence for each outcome (high, moderate, low, or very low quality).”
MethodSun, M., Lai, H., Na, F., Li, S., Qiu, X., Tian, J., Zhang, Z., & Ge, L. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.0023 · Cited by 122
Weibel, S., Schaefer, M. S., Raj, D., Rücker, G., Pace, N. L., Schlesinger, T., Meybohm, P., Kienbaum, P., Eberhart, L. H. J., & Kranke, P. (2020) · Anaesthesia · doi:10.1111/anae.15295 · Cited by 110
“In making inferences regarding the choice of an intervention, recognising the certainty of each comparison is far more valuable than ranking efficacy alone [38].”
BackgroundGarattini, S. K., Jakobsen, J. C., Wetterslev, J., Bertelè, V., Banzi, R., Rath, A., Neugebauer, E. A. M., Laville, M., Masson, Y., Hivert, V., et al. (2016) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2016.03.020 · Cited by 112
Ramaswamy, V. V., Bandyopadhyay, T., Nanda, D., Bandiya, P., Ahmed, J., Garg, A., Roehr, C. C., & Nangia, S. (2021) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2020.6826 · Cited by 114
Schwarzer, G., Carpenter, J. R., & Rücker, G. (2015) · Use R! · doi:10.1007/978-3-319-21416-0_8 · Cited by 109
Nikolakopoulou, A., Mavridis, D., Furukawa, T. A., Cipriani, A., Tricco, A. C., Straus, S. E., Siontis, G. C. M., Egger, M., & Salanti, G. (2018) · BMJ · doi:10.1136/bmj.k585 · Cited by 109
for the FISSH Group (Fluids in Sepsis and Septic Shock), Rochwerg, B., Alhazzani, W., Gibson, A., Ribic, C. M., Sindi, A., Heels-Ansdell, D., Thabane, L., Fox-Robichaud, A., Mbuagbaw, L., et al. (2015) · Intensive Care Medicine · doi:10.1007/s00134-015-3794-1 · Cited by 106
Self-citationDressings and topical agents for treating pressure ulcers
Westby, M. J., Dumville, J. C., Soares, M. O., Stubbs, N. C., & Norman, G. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011947.pub2 · Cited by 117
Ma, H., Lin, Y., Dai, L.-Z., Lin, C., Huang, Y., & Liu, S.-Y. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-061807 · Cited by 125
Roenneberg, C., Sattel, H., Schaefert, R., Henningsen, P., & Hausteiner-Wiehle, C. (2019) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2019.0553 · Cited by 111
Facciorusso, A., Ramai, D., Gkolfakis, P., Khan, S. R., Papanikolaou, I. S., Τriantafyllou, Κ., Tringali, A., Chandan, S., Mohan, B. P., & Adler, D. G. (2021) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2021.09.010 · Cited by 104
Treatment of idiopathic pulmonary fibrosis: a network meta-analysis
Rochwerg, B., Neupane, B., Zhang, Y., Garcia, C. C., Raghu, G., Richeldi, L., Brozek, J., Beyene, J., & Schünemann, H. (2016) · BMC Medicine · doi:10.1186/s12916-016-0558-x · Cited by 101
“The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach specific to NMA served to assess the certainty in the evidence (quality of evidence) associated with specific comparisons, including direct, indirect, and final network meta-analysis estimates [20].”
BackgroundMethodSelf-citationRamaswamy, V. V., More, K., Roehr, C. C., Bandiya, P., & Nangia, S. (2020) · Pediatric Pulmonology · doi:10.1002/ppul.25011 · Cited by 105
“The confidence in the final estimates for all the outcomes was assessed using the GRADE approach as recommended by the GRADE working group.(22) 2.”
MethodProgestogens for preventing miscarriage: a network meta-analysis
Devall, A., Papadopoulou, A., Podesek, M., Haas, D. M., Price, M. J., Coomarasamy, A., & Gallos, I. D. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013792.pub2 · Cited by 111
“We used the GRADE working group's approach (Brignardello-Petersen 2018; Puhan 2014) for rating the certainty of the analysis eLect estimates for all the comparisons and all outcomes.”
MethodVery early versus delayed mobilisation after stroke
Langhorne, P., Collier, J. M., Bate, P. J., Thuy, M. N. T., & Bernhardt, J. A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006187.pub3 · Cited by 103
Twenty years of network meta‐analysis: Continuing controversies and recent developments
Ades, A. E., Welton, N. J., Dias, S., Phillippo, D., & Caldwell, D. M. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1700 · Cited by 115
“67 The rating attached to each indirect estimate would be the lowest of the ratings of its two direct components.”
InfluentialWHO recommendations on uterotonics for postpartum haemorrhage prevention: what works, and which one?
Vogel, J. P., Williams, M. J., Gallos, I. D., Althabe, F., & Oladapo, O. T. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001466 · Cited by 106
Dhiman, R. K., Thumburu, K. K., Verma, N., Chopra, M., Rathi, S., Dutta, U., Singal, A. K., Taneja, S., Duseja, A. K., & Singh, M. (2019) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2019.08.047 · Cited by 100
GRADE guidelines 33: Addressing imprecision in a network meta-analysis
Brignardello‐Petersen, R., Guyatt, G. H., Mustafa, R. A., Chu, D. K., Hultcrantz, M., Schünemann, H. J., & TOMLINSON, G. A. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.07.011 · Cited by 103
Self-citationD'Antonio, F., Marinceu, D., Prasad, S., & Khalil, A. A. (2022) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.26136 · Cited by 99
Aoyama, H., Uchida, K., Aoyama, K., Pechlivanoglou, P., Englesakis, M. F., Yamada, Y., & Fan, E. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.8116 · Cited by 98
Menges, D., Yebyo, H. G., Sivec-Muniz, S., Haile, S. R., Barbier, M. C., Tomonaga, Y., Schwenkglenks, M. M., & Puhan, M. A. (2022) · European Urology Oncology · doi:10.1016/j.euo.2022.04.007 · Cited by 99
Self-citationZhang, Y., Jiang, L., Wang, J., Wang, T., Chien, C., Huang, W., Fu, X., Xiao, Y., Fu, Q., Wang, S., et al. (2022) · Cardiovascular Diabetology · doi:10.1186/s12933-022-01676-5 · Cited by 96
Wang, Y., Flórez, I. D., Morgan, R. L., Foroutan, F., Chang, Y., Crandon, H. N., Zeraatkar, D., Bała, M. M., Mao, R. Q., Tao, B. K.-L., et al. (2023) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.3849 · Cited by 103
Self-citationFoote, C. J., Guyatt, G. H., Vignesh, N. K., Mundi, R., Chaudhry, H., Heels‐Ansdell, D., Thabane, L., Tornetta, P., & Bhandari, M. (2015) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-015-4224-y · Cited by 96
“We dealt with this issue by first identifying the sources of inconsistency (Tables 4 and 5), and second, by using only the highest confidence comparisons as the best estimate of effect, as recommended by the GRADE working group [32].”
MethodSelf-citationVosoughi, K., Atieh, J., Khanna, L., Khoshbin, K., Prokop, L. J., Davitkov, P., Murad, M. H., & Camilleri, M. (2021) · eClinicalMedicine · doi:10.1016/j.eclinm.2021.101213 · Cited by 97
“Certainty in the pooled estimates was assessed using the GRADE approach for network meta-analysis [21].”
MethodSelf-citationJinatongthai, P., Kongwatcharapong, J., Foo, C. Y., Phrommintikul, A., Nathisuwan, S., Thakkinstian, A., Reid, C. M., & Chaiyakunapruk, N. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31441-1 · Cited by 96
Meta-Analysis and Meta-Synthesis Methodologies: Rigorously Piecing Together Research
Leary, H., & Walker, A. E. (2018) · TechTrends · doi:10.1007/s11528-018-0312-7 · Cited by 95
“Finally, results include the ranking probability for each treatment (Puhan et al. 2014) in two different forms (see Table 2 below).”
BackgroundMethodInfluentialManagement of Acute Pain From Non–Low Back, Musculoskeletal Injuries
Busse, J. W., Sadeghirad, B., Oparin, Y., Chen, E., Goshua, A., May, C., Hong, P. J., Agarwal, A., Chang, Y., Ross, S. A., et al. (2020) · Annals of Internal Medicine · doi:10.7326/m19-3601 · Cited by 104
Self-citationWang, Y., Ye, Z., Ge, L., Siemieniuk, R. A. C., Wang, X., Wang, Y., Hou, L., Ma, Z., Agoritsas, T., Vandvik, P. O., et al. (2020) · BMJ · doi:10.1136/bmj.l6744 · Cited by 98
Self-citationManagement of people with early- or very early-stage hepatocellular carcinoma
Majumdar, A., Roccarina, D., Thorburn, D., Davidson, B. R., Tsochatzis, E. A., & Gurusamy, K. S. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011650.pub2 · Cited by 95
Watt, J. A., & Giovane, C. D. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1566-9_12 · Cited by 97
Majzoub, A. M., Nayfeh, T., Barnard, A., Munaganuru, N., Dave, S., Singh, S., Murad, M. H., & Loomba, R. (2021) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.16583 · Cited by 92
Self-citationManeiro, J. R., Souto, A., & Gómez-Reino, J. J. (2017) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2017.02.007 · Cited by 93
Singh, J. A., Hossain, A., Tanjong-Ghogomu, E., Kotb, A., Christensen, R., Mudano, A., Maxwell, L. J., Shah, N. P., Tugwell, P., & Wells, G. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012183 · Cited by 88
Self-citationDulai, P. S., Singh, S., Marquez, E. O., Khera, R., Prokop, L. J., Limburg, P. J., Gupta, S. N., & Murad, M. H. (2016) · BMJ · doi:10.1136/bmj.i6188 · Cited by 92
Self-citationNetwork meta-analysis: the highest level of medical evidence?
Faltinsen, E. G., Storebø, O. J., Jakobsen, J. C., Boesen, K., Lange, T., & Gluud, C. N. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2017-110887 · Cited by 89
“14 Salanti and colleagues further supported the use of a contribution matrix to summarise the quality input of each trial to a given indirect estimate,(14) an approach not advocated by Puhan and colleagues at the time.(12) Recent advances by the GRADE working group further challenged the assumption that quality rating…”
BackgroundMethodInfluentialSystemic Immunomodulatory Treatments for Atopic Dermatitis
Drucker, A. M., Lam, M., Prieto‐Merino, D., Malek, R., Ellis, A. G., Yiu, Z. Z. N., Rochwerg, B. N., Giorgio, S. D., Arents, B. W. M., Mohan, T., et al. (2024) · JAMA Dermatology · doi:10.1001/jamadermatol.2024.2192 · Cited by 102
Gao, Y., Guyatt, G. H., Uyeki, T. M., Liu, M., Chen, Y., Zhao, Y., Shen, Y., Xu, J., Zheng, Q., Li, Z., et al. (2024) · The Lancet · doi:10.1016/s0140-6736(24)01307-2 · Cited by 100
Self-citationGao, Y., Ge, L., Liu, M., Niu, M., Chen, Y., Sun, Y., Chen, J., Yao, L., Wang, Q., Li, Z., et al. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101648 · Cited by 89
Facciorusso, A., Prete, V. D., Buccino, R. V., Valle, N. D., Nacchiero, M. C., Monica, F., Cannizzaro, R., & Muscatiello, N. (2017) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2017.11.007 · Cited by 89
“We used Grading of Recommendations Assessment, Development and Evaluation (GRADE) criteria for network metaanalysis to appraise quality of evidence.(11)”
MethodBrazzelli, M. G., Javanbakht, M., Imamura, M., Hudson, J., Moloney, E., Becker, F., Wallace, S. A., Omar, M. I., Shimonovich, M., MacLennan, G., et al. (2019) · Health Technology Assessment · doi:10.3310/hta23140 · Cited by 89
Florez, I. D., Veroniki, A.-A., Al Khalifah, R., Yepes-Nuñez, J. J., Sierra, J. M., Vernooij, R. W. M., Acosta-Reyes, J., Granados, C. M., Pérez-Gaxiola, G., Cuello-Garcia, C., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0207701 · Cited by 87
Self-citationDesensitizing Toothpastes for Dentin Hypersensitivity: A Network Meta-analysis
Martins, C. C., Firmino, R. T., Riva, J. J., Ge, L., Carrasco‐Labra, A., Brignardello‐Petersen, R., Colunga‐Lozano, L. E., Granville‐Garcia, A. F., Costa, F. O., Yepes-Núñez, J. J., et al. (2020) · Journal of Dental Research · doi:10.1177/0022034520903036 · Cited by 91
Self-citationNavarro‐Tapia, E., Sebastiani, G., Sailer, S., Almeida-Toledano, L., Serra-Delgado, M., García‐Algar, Ó., & Andreu-Férnández, V. (2020) · Nutrients · doi:10.3390/nu12082243 · Cited by 86
“The colonization of pathogenic microbes, which leads to dysbiosis in preterm infants, is linked to the delayed introduction of human milk, early antibiotic intervention, a high rate of caesarean delivery, and total parenteral nutrition [152].”
BackgroundSehmbi, H., Brull, R., Ceballos, K. R., Shah, U. J., Martin, J. E., Tobı́as, A., Solo, K., & Abdallah, F. W. (2020) · Anaesthesia · doi:10.1111/anae.15288 · Cited by 93
Weng, Q., Goh, S.-L., Wu, J., Persson, M. S. M., Wei, J., Sarmanova, A., Li, X., Hall, M., Doherty, M., Jiang, T., et al. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-105898 · Cited by 84
Facciorusso, A., Τriantafyllou, Κ., Murad, M. H., Prokop, L. J., Tziatzios, G., Muscatiello, N., & Singh, S. (2018) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2018.11.058 · Cited by 84
Self-citationTan‐Lim, C. S. C., Esteban‐Ipac, N. A. R., Mantaring, J. B. V., Shih‐Yen, E. C., Recto, M. S. T., Sison, O. T., & Alejandria, M. M. (2020) · Pediatric Allergy and Immunology · doi:10.1111/pai.13305 · Cited by 85
Park, C. H., Jung, J. H., Nam, E., Kim, E. H., Kim, M. G., Kim, J. H., & Park, S. W. (2017) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2017.07.038 · Cited by 82
Yuan, J.-Q., Tsoi, K. K. F., Yang, M. G., Wang, J. Y., Threapleton, D. E., Yang, Z., Zou, B., Mao, C., Tang, J. L., & Chan, F. K. L. (2016) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.13642 · Cited by 86
“The quality of evidence was graded as ‘high’, ‘moderate’, ‘low’ and ‘very low’ based on the fundamental study design and additional methodological factors.(21) For network meta-analysis, we firstly rated the quality of the direct and indirect evidence separately.”
BackgroundMethodHussein, H., Zaccardi, F., Khunti, K., Davies, M. J., Patsko, E., Dhalwani, N. N., Kloecker, D. E., Ioannidou, E., & Gray, L. J. (2020) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.14008 · Cited by 82
“Quality of evidence for the primary outcome was assessed using the GRADE working group approach for the primary outcome.(26) Design-by-treatment models were fitted to assess inconsistency, where possible.”
MethodSiemieniuk, R. A., Bartoszko, J. J., Díaz Martinez, J. P., Kum, E., Qasim, A., Zeraatkar, D., Izcovich, A., Mangala, S., Ge, L., Han, M. A., et al. (2021) · BMJ · doi:10.1136/bmj.n2231 · Cited by 79
“14 Two methodologists with experience in using GRADE rated each domain for each comparison independently, resolving discrepancies by discussion.”
Self-citationSingh, J. A., Hossain, A., Tanjong-Ghogomu, E., Mudano, A., Maxwell, L. J., Buchbinder, R., Lopez-Olivo, M., Suarez-Almazor, M., Tugwell, P., & Wells, G. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012591 · Cited by 65
Self-citationKadakia, A., Dembek, C., Heller, V., Singh, R., Uyei, J., Hagi, K., Nosaka, T., & Loebel, A. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03220-3 · Cited by 79
“Ratings were based on the following domains: study design, risk of bias, inconsistency, indirectness, imprecision, and publication bias [28].”
MethodTherapeutic Options for Neuroendocrine Tumors
Kaderli, R. M., Spanjol, M., Kollár, A., Bütikofer, L., Gloy, V., Dumont, R. A., Seiler, C. A., Christ, E., Radojewski, P., Briel, M., et al. (2019) · JAMA Oncology · doi:10.1001/jamaoncol.2018.6720 · Cited by 83
Treatment of newly diagnosed glioblastoma in the elderly: a network meta-analysis
Hanna, C. R., Lawrie, T. A., Rogozińska, E., Kernohan, A., Jefferies, S., Bulbeck, H., Ali, U. M., Robinson, T., & Grant, R. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013261.pub2 · Cited by 79
Palmer, S. C., Mavridis, D., Johnson, D. W., Tonelli, M. A., Ruospo, M., & Strippoli, G. F. (2020) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2020.02.439 · Cited by 85
Devoe, D. J. A., Peterson, A. M., & Addington, J. (2017) · Schizophrenia Bulletin · doi:10.1093/schbul/sbx139 · Cited by 82
Piechotta, V., Adams, A., Haque, M., Scheckel, B., Kreuzberger, N., Monsef, I., Jordan, K., Kuhr, K., & Skoetz, N. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012775.pub2 · Cited by 79
Facciorusso, A., Crinò, S. F., Ramai, D., Madhu, D., Fugazza, A., Carrara, S., Spadaccini, M., Mangiavillano, B., Gkolfakis, P., Mohan, B. P., et al. (2023) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2023.01.024 · Cited by 80
Devoe, D. J. A., Farris, M. S., Townes, P., & Addington, J. (2018) · Early Intervention in Psychiatry · doi:10.1111/eip.12677 · Cited by 86
“In the NMA, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach was used to evaluate the quality of evidence associated with the results in the NMA at each time point (Puhan et al., 2014).”
MethodFong, K. M., Au, S. Y., & Ng, G. W. Y. (2019) · Critical Care · doi:10.1186/s13054-019-2596-1 · Cited by 79
“When both direct and indirect evidence were available, we chose the higher of the two quality ratings for the NMA estimate [17].”
BackgroundMethodRamaswamy, V. V., Bandyopadhyay, T., Nanda, D., Bandiya, P., More, K., Oommen, V. I., & Gupta, A. (2020) · Pediatric Pulmonology · doi:10.1002/ppul.25007 · Cited by 79
“Surface under the cumulative ranking curve (SUCRA) plots were used to rank the different NRS modalities for all the outcomes.19 The quality of evidence was graded using the GRADE approach for NMA.20”
MethodInfluentialSingh, J. A., Hossain, A., Mudano, A., Tanjong-Ghogomu, E., Suarez-Almazor, M., Buchbinder, R., Maxwell, L. J., Tugwell, P., & Wells, G. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012657 · Cited by 71
Self-citationBruder, A. M., Shields, N., Dodd, K., & Taylor, N. F. (2017) · Journal of physiotherapy · doi:10.1016/j.jphys.2017.08.009 · Cited by 78
Sekercioglu, N., Thabane, L., Martinez, J. P. D., NESRALLAH, G. E., Longo, C. J., Busse, J. W., Akhtar‐Danesh, N., Agarwal, A., Khalifah, R. A. A., Iorio, A., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0156891 · Cited by 80
Self-citationBest, L. M., Freeman, S. C., Sutton, A. J., Cooper, N. J., Tng, E.-L., Csenar, M., Hawkins, N., Pavlov, C. S., Davidson, B. R., Thorburn, D., et al. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013103.pub2 · Cited by 79
“Puhan 2014 Puhan MA, Schünemann HJ, Murad MH, Li T, BrignardelloPetersen R, Singh JA, et al. A GRADE Working Group approach for rating the quality of treatment eEect estimates from network meta-analysis.”
InfluentialInterventions for the Management of Computer Vision Syndrome
Singh, S., McGuinness, M. B., Anderson, A. J., & Downie, L. E. (2022) · Ophthalmology · doi:10.1016/j.ophtha.2022.05.009 · Cited by 87
Castellini, G., Pillastrini, P., Vanti, C., Bargeri, S., Giagio, S., Bordignon, E., Fasciani, F., Marzioni, F., Innocenti, T., Chiarotto, A., et al. (2022) · Journal of physiotherapy · doi:10.1016/j.jphys.2022.09.007 · Cited by 95
Nagendrababu, V., Pulikkotil, S. J., Suresh, A., Veettil, S. K., Bhatia, S., & Setzer, F. C. (2019) · International Endodontic Journal · doi:10.1111/iej.13072 · Cited by 79
“The quality of evidence was rated as ‘high’, ‘moderate’, ‘low’ and ‘very low’ (Puhan et al. 2014).”
Dayyani, M., Sadeghirad, B., Grotta, J. C., Zabihyan, S., Ahmadvand, S., Wang, Y., Guyatt, G. H., & Amin‐Hanjani, S. (2022) · Stroke · doi:10.1161/strokeaha.121.035699 · Cited by 80
Self-citationAkenroye, A. T., Lassiter, G., Jackson, J. W., Keet, C. A., Segal, J. B., Alexander, G. C., & Hong, H. (2022) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2022.05.024 · Cited by 79
Cornu, J., Zantek, P. F., Burtt, G., Martin, C., Martin, A. E., Springate, C., & Chughtai, B. I. (2023) · European Urology · doi:10.1016/j.eururo.2023.02.028 · Cited by 76
Delivery Room Interventions for Hypothermia in Preterm Neonates
Abiramalatha, T., Ramaswamy, V. V., Bandyopadhyay, T., Pullattayil, A. K., Thanigainathan, S., Trevisanuto, D., & Roehr, C. C. (2021) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2021.0775 · Cited by 79
Challoumas, D., Pedret, C., Biddle, M., Ng, N. Y. B., Kirwan, P., Cooper, B., Nicholas, P., Wilson, S., Clifford, C., & Millar, N. L. (2021) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2021-001110 · Cited by 75
“8 9 Strength of evidence assessment was performed by DC and MB independently and any disagreements were resolved by discussion and involvement of a third assessor (NLM).”
Interventions for heavy menstrual bleeding; overview of Cochrane reviews and network meta-analysis
Rodriguez, M. B., Dias, S., Jordan, V., Lethaby, A. E., Lensen, S. F., WISE, M. R., Wilkinson, J., Brown, J., & Farquhar, C. M. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013180.pub2 · Cited by 82
Darling, K. E., & Sato, A. F. (2017) · Childhood Obesity · doi:10.1089/chi.2017.0038 · Cited by 75
Pharmacological interventions for non-alcohol related fatty liver disease (NAFLD)
Lombardi, R., Onali, S., Thorburn, D., Davidson, B. R., Gurusamy, K. S., & Tsochatzis, E. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011640.pub2 · Cited by 74
Kim, W. H., Hur, M. S., Park, S.-K., Yoo, S., Lim, T. Y., Yoon, H., Kim, J., & Bahk, J.-H. (2018) · International Journal of Obstetric Anesthesia · doi:10.1016/j.ijoa.2018.09.012 · Cited by 88
Paglia, M. D. G., Silva, M. T., Lopes, L. C., Barberato-Filho, S., Mazzei, L. G., Abe, F. C., & Bergamaschi, C. D. C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248866 · Cited by 81
“We followed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to appraise the confidence in estimates derived from network meta-analysis of outcomes [34, 37].”
MethodPhillippo, D., Dias, S., Welton, N. J., Caldwell, D. M., Taske, N., & Ades, A. E. (2019) · Annals of Internal Medicine · doi:10.7326/m18-3542 · Cited by 71
“GRADE NMA then combines these judgements in a subjective manner to reach an overall judgement, whereas Salanti GRADE uses an objective statistical rationale to reach an overall judgement.”
InfluentialYou, R., Cao, Y.-S., Huang, P., Chen, L., Yang, Q., Liu, Y.-P., Zou, X., Zhang, Y.-N., Jiang, R., Zhang, M., et al. (2017) · Theranostics · doi:10.7150/thno.21815 · Cited by 72
Methods for managing miscarriage: a network meta-analysis
Ghosh, J., Papadopoulou, A., Devall, A., Jeffery, H. C., Beeson, L. E., Do, V., Price, M. J., Tobı́as, A., Tunçalp, Ö., Lavelanet, A. F., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012602.pub2 · Cited by 74
Topical Fluoride to Prevent Root Caries: Systematic Review with Network Meta-analysis
Zhang, J., Sardana, D., Li, K. Y., Leung, K., & Lo, E. C. M. (2020) · Journal of Dental Research · doi:10.1177/0022034520906384 · Cited by 80
“For indirect comparison, intransitivity, imprecision, and the lowest-certainty ratings for the direct comparisons formed the first-order loop (Puhan et al. 2014).”
BackgroundComparison of Antiviral Agents for Seasonal Influenza Outcomes in Healthy Adults and Children
Liu, J.-W., Lin, S.-H., Wang, L.-C., Chiu, H.-Y., & Lee, J. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.19151 · Cited by 73
Vandvik, P. O., Brignardello-Petersen, R., & Guyatt, G. H. (2016) · BMC Medicine · doi:10.1186/s12916-016-0596-4 · Cited by 69
“Authors have already presented two systems based on GRADE principles for making quality of evidence ratings in NMAs [21, 22].”
InfluentialSelf-citation
Metadata
- Authors
- Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group
- Affiliations
- University of Zurich (CH)
- Mayo Clinic (US)
- Johns Hopkins University (US)
- University of Alabama (US)
- Maastricht University (NL)
- McMaster University (CA)
- Corresponding author
- Puhan, M. A.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2014-09-24
- Volume
- 349
- Issue
- sep24 5
- Pages
- g5630-g5630
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155798268
- Cited by
- 1,812
- References
- 39
- Citations by year
- 2026: 160 · 2025: 177 · 2024: 216 · 2023: 175 · 2022: 205 · 2021: 276 · 2020: 201 · 2019: 128 · 2018: 96 · 2017: 72 · 2016: 63 · 2015: 36 · 2014: 4
- FWCI
- 57.15
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-06-17)
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- network meta-analysis, quality of evidence, GRADE, treatment effect estimates, comparative effectiveness
- MeSH terms
- Humans, Quality Assurance, Health Care, Meta-Analysis as Topic, Practice Guidelines as Topic, Evidence-Based Medicine, Guideline Adherence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review