GRADE guidelines: 8. Rating the quality of evidence—indirectness
Open accessAbstract
Direct evidence comes from research that directly compares the interventions in which we are interested when applied to the populations in which we are interested and measures outcomes important to patients. Evidence can be indirect in one of four ways. First, patients may differ from those of interest (the term applicability is often used for this form of indirectness). Secondly, the intervention tested may differ from the intervention of interest. Decisions regarding indirectness of patients and interventions depend on an understanding of whether biological or social factors are sufficiently different that one might expect substantial differences in the magnitude of effect. Thirdly, outcomes may differ from those of primary interest-for instance, surrogate outcomes that are not themselves important, but measured in the presumption that changes in the surrogate reflect changes in an outcome important to patients. A fourth type of indirectness, conceptually different from the first three, occurs when clinicians must choose between interventions that have not been tested in head-to-head comparisons. Making comparisons between treatments under these circumstances requires specific statistical methods and will be rated down in quality one or two levels depending on the extent of differences between the patient populations, co-interventions, measurements of the outcome, and the methods of the trials of the candidate interventions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The overall quality of evidence was graded using the GRADE approach, considering study limitations, inconsistency, indirectness, imprecision and publication bias.
“The overall quality of evidence was graded using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach, considering study limitations, inconsistency, indirectness, imprecision and publication bias [40–44].”
From Efficacy of manual therapy by different healthcare professionals on pain and function in temporomandibular disorders: a systematic review of randomized controlled trials (2026) · cites this paper as [44]
The Introduction describes GRADE rating down evidence for bias, imprecision, inconsistency, and indirectness. The paper also discusses risk of bias from study limitations, but the passages found do not identify publication bias as a consideration; the Introduction’s general reference to “bias” does not establish that specific item. No page numbers are available in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measuredQuality of evidence in included systematic reviews was assessed by outcome using the GRADE system criteria.
“Quality of evidence in included systematic reviews was assessed by outcome using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) system criteria [17-27].”
From Effects of Home Telemonitoring Interventions on Patients With Chronic Heart Failure: An Overview of Systematic Reviews (2015) · cites this paper as [24]
RCTs are first categorised as high-quality evidence in the GRADE system, while observational studies are classified as low-quality data supporting estimates of intervention effects.
“RCTs are first categorised as high-quality evidence in the GRADE system, while observational studies are classified as low-quality data supporting estimates of intervention effects.”
The GRADE approach was used to determine the quality of evidence.
“To determine the quality of evidence, the GRADE approach was used [29–36].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Assessing Risk of Bias in Included Studies
Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch8 · Cited by 3,396 · see citing works
Cipriani, A., Furukawa, T. A., Salanti, G., Geddes, J. R., Higgins, J. P., Churchill, R., Watanabe, N., Nakagawa, A., Omori, I. M., McGuire, H., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60046-5 · Cited by 1,654 · see citing works
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · see citing works
Flagged first, then by citations
Research works printed but not linked: 16
- Schunemann HJ, Hill SR, Kakad M, Bellamy R, Uyeki TM, Hayden FG, et al. WHO Rapid Advice Guidelines for pharmacolog- ical management of sporadic human infection with avian influenza A (H5N1) virus. Lancet Infect Dis 2007;7(1):21e31.
- Rothwell PM. External validity of randomised controlled trials: ''to whom do the results of this trial apply?''. Lancet 2005;365(9453):82e93.
- Hassiotis A, Hall I. Behavioural and cognitive-behavioural interven- tions for outwardly-directed aggressive behaviour in people with learning disabilities. Cochrane Database Syst Rev 2004;1: CD003406. DOI:10.1002/14651858.CD003406.pub2. · DOI
- Akl EA, Terrenato I, Barba M, Sperati F, Sempos EV, Muti P, et al. Low-molecular-weight heparin vs unfractionated heparin for perioperative thromboprophylaxis in patients with cancer: a system- atic review and meta-analysis. Arch Intern Med 2008;168:1261e9.
- Miksad RA, Zietemann V, Gothe R, Schwarzer R, Conrads-Frank A, Schnell-Inderst P, et al. Progression-free survival as a surrogate end- point in advanced breast cancer. Int J Technol Assess Health Care 2008;24(4):371e83.
- Lassere MN, Johnson KR, Boers M, Tugwell P, Brooks P, Simon L, et al. Definitions and validation criteria for biomarkers and surrogate endpoints: development and testing of a quantitative hierarchical levels of evidence schema. J Rheumatol 2007;34: 607e15.
- Sebba A. Comparing non-vertebral fracture risk reduction with oste- oporosis therapies: looking beneath the surface. Osteoporos Int 2009;20:675e86.
- Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG, et al. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 2009;338:b1147.
- Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol 1997;50:683e91.
- Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D'Amico R, et al. Indirect comparisons of competing interventions. Health Technol Assess 2005;9(26):1e134, iiieiv.
- de Lorenzo F, Feher M, Martin J, Collot-Teixeira S, Dotsenko O, McGregor JL. Statin therapy-evidence beyond lipid lowering contrib- uting to plaque stability. Curr Med Chem 2006;13:3385e93.
- Fisman EZ, Adler Y, Tenenbaum A. Statins research unfinished saga: desirability versus feasibility. Cardiovasc Diabetol 2005;4(1):8.
- Wanner C, Krane V, M€ arz W, Olschewski M, Mann JF, Ruf G, et al. Atorvastatin in patients with type 2 diabetes mellitus undergoing hemodialysis. N Engl J Med 2005;353(3):238e48.
- Bucher H, Kunz R, Cook D, Holbrook A, Guyatt G. Surrogate out- comes. In: Guyatt G, Rennie D, Meade M, Cook D, editors. The users' guides to the medical literature: a manual for evidence-based clinical practice. New York, NY: McGraw-Hill; 2008.
- Kunz R, Bucher H, McAlister F, Holbrook A, Guyatt G. Drug class effects. In: Guyatt G, Rennie D, Meade M, Cook D, editors. The users' guides to the medical literature: a manual for evidence-based clinical practice. New York, NY: McGraw-Hill; 2008.
- Lim E, Ali Z, Ali A, Routledge T, Edmonds L, Altman DG, et al. Indirect comparison meta-analysis of aspirin therapy after coronary surgery. BMJ 2003;327(7427):1309.
Cited by
Sorted by citations
Hao (2015) · doi:10.1002/14651858 · Cited by 11,321 · see citing works
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285 · see citing works
Self-citationAntithrombotic Therapy for VTE Disease
Kearon, C., Akl, E. A., Ornelas, J., Blaivas, A., Jimenez, D., Bounameaux, H., Huisman, M., King, C. S., Morris, T. A., Sood, N., et al. (2016) · Chest · doi:10.1016/j.chest.2015.11.026 · Cited by 4,830 · see citing works
Self-citationJapanese Society for Cancer of the Colon and Rectum, Watanabe, T., Muro, K., Ajioka, Y., Hashiguchi, Y., Ito, Y., Saito, Y., Hamaguchi, T., Ishida, H., Ishiguro, M., et al. (2018) · International Journal of Clinical Oncology · doi:10.1007/s10147-017-1101-6 · Cited by 2,467 · see citing works
GRADE guidelines: 2. Framing the question and deciding on important outcomes
Guyatt, G. H., Oxman, A. D., Kunz, R., Atkins, D., Brozek, J., Vist, G., Alderson, P., Glasziou, P., Falck-Ytter, Y., & Schünemann, H. J. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.012 · Cited by 1,911 · see citing works
Self-citationCINeMA: an approach for assessing confidence in the results of a network meta-analysis
Nikolakopoulou, A., Higgins, J. P. T., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., & Salanti, G. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003082 · Cited by 1,935 · see citing works
Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812 · see citing works
Self-citationEvaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · see citing works
Psychological therapies for people with borderline personality disorder.
Stoffers-Winterling, J. M., Völlm, B. A., Rücker, G., Timmer, A., Huband, N., & Lieb, K. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005652.pub2 · Cited by 895 · see citing works
GRADE guidelines: 12. Preparing Summary of Findings tables—binary outcomes
Guyatt, G. H., Oxman, A. D., Santesso, N., Helfand, M., Vist, G., Kunz, R., Brozek, J., Norris, S., Meerpohl, J., Djulbegovic, B., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.01.012 · Cited by 836 · see citing works
Self-citationGuyatt, G., Oxman, A. D., Sultan, S., Brozek, J., Glasziou, P., Alonso-Coello, P., Atkins, D., Kunz, R., Montori, V., Jaeschke, R., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.01.006 · Cited by 762 · see citing works
Self-citationSystematic review and meta-analysis of psychological therapies for children with chronic pain
Fisher, E., Heathcote, L., Palermo, T. M., de C Williams, A. C., Lau, J., & Eccleston, C. (2014) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsu008 · Cited by 710 · see citing works
Early palliative care for adults with advanced cancer
Haun, M. W., Estel, S., Rücker, G., Friederich, H.-C., Villalobos, M., Thomas, M., & Hartmann, M. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011129.pub2 · Cited by 757 · see citing works
The Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis
Liu, Q., Peng, W., Zhang, F., Hu, R., Li, Y., & Yan, W. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4807 · Cited by 691 · see citing works
Systematic review of physical activity and health in the early years (aged 0-4 years)
Timmons, B. W., LeBlanc, A. G., Carson, V., Connor Gorber, S., Dillman, C., Janssen, I., Kho, M. E., Spence, J. C., Stearns, J. A., & Tremblay, M. S. (2012) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/h2012-070 · Cited by 626 · see citing works
Saunders, T. J., Gray, C. E., Poitras, V. J., Chaput, J.-P., Janssen, I., Katzmarzyk, P. T., Olds, T., Connor Gorber, S., Kho, M. E., Sampson, M., et al. (2016) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2015-0626 · Cited by 589 · see citing works
“And for indirectness, we considered all studies reporting the outcome, and examined whether studies included the populations, interventions, comparators, or outcomes of interest (Guyatt et al. 2011c).”
BackgroundMethodInfluentialMachado, G. C., Maher, C. G., Ferreira, P. H., Pinheiro, M. B., Lin, C.-W. C., Day, R. O., McLachlan, A. J., & Ferreira, M. L. (2015) · BMJ · doi:10.1136/bmj.h1225 · Cited by 580 · see citing works
Chiavaroli, L., Viguiliouk, E., Nishi, S., Blanco Mejia, S., Rahelić, D., Kahleová, H., Salas-Salvadó, J., Kendall, C., & Sievenpiper, J. (2019) · Nutrients · doi:10.3390/nu11020338 · Cited by 548 · see citing works
Homocysteine lowering interventions for preventing cardiovascular events
Martí-Carvajal, A. J., Solà, I., Lathyris, D., & Salanti, G. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006612.pub2 · Cited by 520 · see citing works
Murad, M. H., Montori, V. M., Ioannidis, J. P. A., Jaeschke, R., Devereaux, P. J., Prasad, K., Neumann, I., Carrasco-Labra, A., Agoritsas, T., Hatala, R., et al. (2014) · JAMA · doi:10.1001/jama.2014.5559 · Cited by 454 · see citing works
Self-citationPsychological therapies for people with borderline personality disorder
Storebø, O. J., Stoffers-Winterling, J. M., Völlm, B. A., Kongerslev, M. T., Mattivi, J. T., Jørgensen, M. S., Faltinsen, E., Todorovac, A., Sales, C. P., Callesen, H. E., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012955.pub2 · Cited by 450 · see citing works
Rooney, A. A., Boyles, A. L., Wolfe, M. S., Bucher, J. R., & Thayer, K. A. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307972 · Cited by 434 · see citing works
Psychological interventions to foster resilience in healthcare professionals
Kunzler, A. M., Helmreich, I., Chmitorz, A., König, J., Binder, H., Wessa, M., & Lieb, K. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012527.pub2 · Cited by 390 · see citing works
Schwingshackl, L., Knüppel, S., Schwedhelm, C., Hoffmann, G., Missbach, B., Stelmach-Mardas, M., Dietrich, S., Eichelmann, F., Kontopanteils, E., Iqbal, K., et al. (2016) · Advances in Nutrition · doi:10.3945/an.116.013052 · Cited by 360 · see citing works
Kitsiou, S., Paré, G., & Jaana, M. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4174 · Cited by 327 · see citing works
Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)
Storebø, O. J., Ramstad, E., Krogh, H. B., Nilausen, T. D., Skoog, M., Holmskov, M., Rosendal, S., Groth, C., Magnusson, F. L., Moreira-Maia, C. R., et al. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009885.pub2 · Cited by 347 · see citing works
Schoenfeld, B. J., Ogborn, D., & Krieger, J. W. (2016) · Sports Medicine · doi:10.1007/s40279-016-0543-8 · Cited by 308 · see citing works
Grading the strength of a body of evidence when assessing health care interventions: an EPC update
Berkman, N. D., Lohr, K. N., Ansari, M. T., Balk, E. M., Kane, R., McDonagh, M., Morton, S. C., Viswanathan, M., Bass, E. B., Butler, M., et al. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.023 · Cited by 279 · see citing works
Maternal Anemia and Low Birth Weight: A Systematic Review and Meta-Analysis
Figueiredo, A. C. M. G., Gomes-Filho, I. S., Silva, R. B., Pereira, P. P. S., Mata, F. A. F. D., Lyrio, A. O., Souza, E. S., Cruz, S. S., & Pereira, M. G. (2018) · Nutrients · doi:10.3390/nu10050601 · Cited by 273 · see citing works
Momma, H., Kawakami, R., Honda, T., & Sawada, S. S. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-105061 · Cited by 265 · see citing works
Using systematic reviews in guideline development: The GRADE approach
Zhang, Y., Akl, E. A., & Schünemann, H. J. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1313 · Cited by 260 · see citing works
Self-citationSchünemann, H. J., Tugwell, P., Reeves, B. C., Akl, E. A., Santesso, N., Spencer, F. A., Shea, B., Wells, G., & Helfand, M. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1078 · Cited by 248 · see citing works
Self-citationSupplemental vitamins and minerals for CVD prevention and treatment
Jenkins, D. J. A., Spence, J. D., Giovannucci, E. L., Kim, Y.-I., Josse, R., Vieth, R., Blanco Mejia, S., Viguiliouk, E., Nishi, S., Sahye-Pudaruth, S., et al. (2018) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.04.020 · Cited by 251 · see citing works
Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Ansari, M., Akl, E. A., et al. (2019) · Environment International · doi:10.1016/j.envint.2018.11.004 · Cited by 248 · see citing works
Self-citationLam, J., Koustas, E., Sutton, P., Johnson, P. I., Atchley, D. S., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307923 · Cited by 214 · see citing works
Viguiliouk, E., Kendall, C. W., Kahleová, H., Rahelić, D., Salas-Salvadó, J., Choo, V. L., Mejia, S. B., Stewart, S. E., Leiter, L. A., Jenkins, D. J., et al. (2019) · Clinical Nutrition · doi:10.1016/j.clnu.2018.05.032 · Cited by 201 · see citing works
Lang, J. J., Belanger, K., Poitras, V., Janssen, I., Tomkinson, G. R., & Tremblay, M. S. (2018) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2017.08.002 · Cited by 194 · see citing works
Exercise for treating patellofemoral pain syndrome
van der Heijden, R. A., Lankhorst, N. E., van Linschoten, R., Bierma-Zeinstra, S. M., & van Middelkoop, M. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010387.pub2 · Cited by 185 · see citing works
Hooijmans, C. R., de Vries, R. B. M., Ritskes-Hoitinga, M., Rovers, M. M., Leeflang, M. M., IntHout, J., Wever, K. E., Hooft, L., de Beer, H., Kuijpers, T., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0187271 · Cited by 176 · see citing works
Self-citationHall, A., Richmond, H., Copsey, B., Hansen, Z., Williamson, E., Jones, G., Fordham, B., Cooper, Z., & Lamb, S. (2018) · Disability and Rehabilitation · doi:10.1080/09638288.2016.1236155 · Cited by 163 · see citing works
Munkholm, K., Paludan-Müller, A. S., & Boesen, K. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024886 · Cited by 151 · see citing works
He, J., Morales, D. R., & Guthrie, B. (2020) · Trials · doi:10.1186/s13063-020-4139-0 · Cited by 147 · see citing works
Storebø, O. J., Skoog, M., Damm, D., Thomsen, P. H., Simonsen, E., & Gluud, C. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008223.pub2 · Cited by 149 · see citing works
Farrell, B., Pottie, K., Rojas-Fernandez, C. H., Bjerre, L. M., Thompson, W., & Welch, V. (2016) · PLOS ONE · doi:10.1371/journal.pone.0161248 · Cited by 134 · see citing works
Li, S. S., Blanco Mejia, S., Lytvyn, L., Stewart, S. E., Viguiliouk, E., Ha, V., de Souza, R. J., Leiter, L. A., Kendall, C. W. C., Jenkins, D. J. A., et al. (2017) · Journal of the American Heart Association · doi:10.1161/jaha.117.006659 · Cited by 139 · see citing works
Thomas, S.-M., Jeyaraman, M., Hodge, W. G., Hutnik, C., Costella, J., & Malvankar-Mehta, M. S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0113779 · Cited by 138 · see citing works
Evidence on Virtual Reality-Based Therapies for Psychiatric Disorders: Meta-Review of Meta-Analyses
Dellazizzo, L., Potvin, S., Luigi, M., & Dumais, A. (2020) · Journal of Medical Internet Research · doi:10.2196/20889 · Cited by 135 · see citing works
Uhlig, K., Leff, B., Kent, D., Dy, S., Brunnhuber, K., Burgers, J. S., Greenfield, S., Guyatt, G., High, K., Leipzig, R., et al. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-013-2659-y · Cited by 158 · see citing works
Self-citationDellazizzo, L., Potvin, S., Giguère, S., & Dumais, A. (2022) · Addiction · doi:10.1111/add.15764 · Cited by 120 · see citing works
Lindström, L., Lindström, E., Nilsson, M., & Höistad, M. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.02.012 · Cited by 122 · see citing works
Philipp, R., Kriston, L., Lanio, J., Kühne, F., Härter, M., Moritz, S., & Meister, R. (2019) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2345 · Cited by 114 · see citing works
Tsilas, C. S., de Souza, R. J., Mejia, S. B., Mirrahimi, A., Cozma, A. I., Jayalath, V. H., Ha, V., Tawfik, R., Di Buono, M., Jenkins, A. L., et al. (2017) · Canadian Medical Association Journal · doi:10.1503/cmaj.160706 · Cited by 109 · see citing works
Homocysteine-lowering interventions for preventing cardiovascular events
Martí-Carvajal, A. J., Solà, I., & Lathyris, D. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006612.pub4 · Cited by 113 · see citing works
Kahleova, H., Salas-Salvadó, J., Rahelić, D., Kendall, C. W., Rembert, E., & Sievenpiper, J. L. (2019) · Nutrients · doi:10.3390/nu11092209 · Cited by 114 · see citing works
Lee, K. W., Loh, H. C., Ching, S. M., Devaraj, N. K., & Hoo, F. K. (2020) · Nutrients · doi:10.3390/nu12061604 · Cited by 105 · see citing works
Viguiliouk, E., Blanco Mejia, S., Kendall, C. W. C., & Sievenpiper, J. L. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13312 · Cited by 101 · see citing works
Montgomery, P., Underhill, K., Gardner, F., Operario, D., & Mayo-Wilson, E. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.03.006 · Cited by 92 · see citing works
Abdollahi, S., Soltani, S., de Souza, R. J., Forbes, S. C., Toupchian, O., & Salehi-Abargouei, A. (2021) · Advances in Nutrition · doi:10.1093/advances/nmaa156 · Cited by 91 · see citing works
Metformin or oral contraceptives for adolescents with polycystic ovarian syndrome: a meta-analysis
Al Khalifah, R. A., Florez, I. D., Dennis, B., Thabane, L., & Bassilious, E. (2016) · PEDIATRICS · doi:10.1542/peds.2015-4089 · Cited by 91 · see citing works
Welch, V. A., Akl, E. A., Pottie, K., Ansari, M. T., Briel, M., Christensen, R., Dans, A., Dans, L., Eslava-Schmalbach, J., Guyatt, G., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.01.015 · Cited by 88 · see citing works
Self-citationSuzumura, E. A., Bersch-Ferreira, Â. C., Torreglosa, C. R., da Silva, J. T., Coqueiro, A. Y., Kuntz, M. G. F., Chrispim, P. P., Weber, B., & Cavalcanti, A. B. (2019) · Nutrition Reviews · doi:10.1093/nutrit/nuz001 · Cited by 85 · see citing works
El Dib, R., Periyasamy, A. G., de Barros, J. L., França, C. G., Senefonte, F. L., Vesentini, G., Alves, M. G. O., Rodrigues, J. V. D. S., Gomaa, H., Gomes Júnior, J. R., et al. (2021) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2021.07.027 · Cited by 75 · see citing works
Ayoub-Charette, S., Liu, Q., Khan, T. A., Au-Yeung, F., Blanco Mejia, S., de Souza, R. J., Wolever, T. M., Leiter, L. A., Kendall, C., & Sievenpiper, J. L. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024171 · Cited by 67 · see citing works
A systematic review of vasopressor blood pressure targets in critically ill adults with hypotension.
Hylands, M., Moller, M. H., Asfar, P., Toma, A., Frenette, A. J., Beaudoin, N., Belley-Côté, É., D’Aragon, F., Laake, J. H., Siemieniuk, R. A., et al. (2017) · Canadian Journal of Anesthesia/Journal canadien d'anesthésie · doi:10.1007/s12630-017-0877-1 · Cited by 69 · see citing works
Self-citationHersi, M., Stevens, A., Quach, P., Hamel, C., Thavorn, K., Garritty, C., Skidmore, B., Vallenas, C., Norris, S. L., Egger, M., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0140290 · Cited by 64 · see citing works
Self-citationBobos, P., Nazari, G., Szekeres, M., Lalone, E. A., Ferreira, L., & MacDermid, J. C. (2019) · Journal of Hand Therapy · doi:10.1016/j.jht.2018.09.012 · Cited by 64 · see citing works
Meta-review on the efficacy of psychological therapies for the treatment of substance use disorders
Dellazizzo, L., Potvin, S., Giguère, S., Landry, C., Léveillé, N., & Dumais, A. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115318 · Cited by 55 · see citing works
Austin, T. M., Richter, R. R., & Sebelski, C. A. (2014) · Physical Therapy · doi:10.2522/ptj.20130627 · Cited by 56 · see citing works
Hangaard, S., Laursen, S. H., Andersen, J. D., Kronborg, T., Vestergaard, P., Hejlesen, O., & Udsen, F. W. (2023) · Journal of Diabetes Science and Technology · doi:10.1177/19322968211064633 · Cited by 55 · see citing works
Li, R., Li, Y., Kong, Y., Li, H., Hu, D., Fu, C., & Wei, Q. (2024) · Journal of Medical Internet Research · doi:10.2196/45406 · Cited by 48 · see citing works
“The quality of RCTs is initially considered high and then downgraded based on risk of bias, imprecision, inconsistency, indirectness, and publication bias [45-49].”
Sargeant, J. M., & O'Connor, A. M. (2014) · Zoonoses and Public Health · doi:10.1111/zph.12124 · Cited by 42 · see citing works
Height outcome of the recombinant human growth hormone treatment in Turner syndrome: a meta-analysis
Li, P., Cheng, F., & Xiu, L. (2018) · Endocrine Connections · doi:10.1530/ec-18-0115 · Cited by 41 · see citing works
Control interventions in randomised trials among people with mental health disorders
Faltinsen, E., Todorovac, A., Staxen Bruun, L., Hróbjartsson, A., Gluud, C., Kongerslev, M. T., Simonsen, E., & Storebø, O. J. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000050.pub2 · Cited by 40 · see citing works
The Evolution of the AASM Clinical Practice Guidelines: Another Step Forward
Morgenthaler, T. I., Deriy, L., Heald, J. L., & Thomas, S. M. (2016) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.5412 · Cited by 38 · see citing works
Kiratipaisarl, W., Surawattanasakul, V., & Sirikul, W. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06195-3 · Cited by 37 · see citing works
“Eight domains were assessed: inconsistency [40], indirectness [41], imprecision [42], risk of bias [33], publication bias, dose-response gradient, magnitude of association, and presence of residual confounding [43].”
Akl, E. A., Morgan, R. L., Rooney, A. A., Beverly, B., Katikireddi, S. V., Agarwal, A., Alper, B. S., Alva-Diaz, C., Amato, L., Ansari, M. T., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.037 · Cited by 34 · see citing works
Self-citationGrant, S., Azhar, G., Han, E., Booth, M., Motala, A., Larkin, J., & Hempel, S. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003822 · Cited by 31 · see citing works
Tsima, Setlhare, V., & Nkomazana, O. (2016) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s112466 · Cited by 30 · see citing works
Sacrocolpopexy compared with transvaginal mesh surgery: A systematic review and meta-analysis
Zhang, C., Sun, Z., Yang, J., Xu, T., Zhu, L., & Lang, J. (2021) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16324 · Cited by 26 · see citing works
GIUSTI, E. M., SPATOLA, C. A., BRUNANI, A., KUMBHARE, D., ORAL, A., ILIEVA, E., KIEKENS, C., PIETRABISSA, G., MANZONI, G. M., IMAMURA, M., et al. (2020) · European Journal of Physical and Rehabilitation Medicine · doi:10.23736/s1973-9087.20.06232-2 · Cited by 25 · see citing works
Kremer, F., Ditzen, B., & Wischmann, T. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282065 · Cited by 22 · see citing works
RetractedShahinfar, H., Bazshahi, E., Amini, M. R., Payandeh, N., Pourreza, S., Noruzi, Z., & Shab‐Bidar, S. (2021) · Phytotherapy Research · doi:10.1002/ptr.7247 · Cited by 18 · see citing works
Park, S., Tang, A., Pollock, C., & Sakakibara, B. M. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-055527 · Cited by 12 · see citing works
Placebo, usual care and wait-list interventions for all mental health disorders
Faltinsen, E., Todorovac, A., Hróbjartsson, A., Gluud, C., Kongerslev, M. T., Simonsen, E., & Storebø, O. J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000050 · Cited by 9 · see citing works
Gates, M., Gates, A., Pieper, D., Fernandes, R., Tricco, A., Moher, D., Brennan, S. E., Li, T., Pollock, M., Lunny, C., et al. (2022) · doi:10.31222/osf.io/23b6j · Cited by 3 · see citing works
Giusti, E. M., Spatola, C. A. M., Brunani, A., Kumbhare, D., Oral, A., Ilieva, E., Kiekens, C., Pietrabissa, G., Manzoni, G. M., Imamura, M., et al. (2020) · The Journal of the International Society of Physical and Rehabilitation Medicine · doi:10.4103/jisprm.jisprm_6_20 · Cited by 2 · see citing works
Chiaramonte, R., Ferrillo, M., de Sire, A., & Vecchio, M. (2026) · Journal of Manual & Manipulative Therapy · doi:10.1080/10669817.2026.2629282
“The overall quality of evidence was graded using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach, considering study limitations, inconsistency, indirectness, imprecision and publication bias [40–44].”
- Authors
- Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., Akl, E. A., Post, P. N., Norris, S., Meerpohl, J., Shukla, V. K., Nasser, M., & Schünemann, H. J.
- Affiliations
- McMaster University (CA)
- Nasjonalt Kunnskapssenter for Helsetjenesten (NO)
- University Hospital of Basel (CH)
- London School of Hygiene & Tropical Medicine (GB)
- Oregon Health & Science University (US)
- University of Oxford (GB)
- University at Buffalo, State University of New York (US)
- Dutch Institute for Healthcare Improvement (NL)
- University Medical Center Freiburg (DE)
- Canada's Drug Agency (CA)
- Institute for Quality and Efficiency in Health Care (DE)
- Corresponding author
- Guyatt, Gordon H.
- ORCID
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2011-12
- Volume
- 64
- Issue
- 12
- Pages
- 1303-1310
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2009326492
- Cited by
- 1,843
- References
- 4
- Citations by year
- 2026: 108 · 2025: 136 · 2024: 171 · 2023: 152 · 2022: 173 · 2021: 151 · 2020: 180 · 2019: 113 · 2018: 125 · 2017: 122 · 2016: 131 · 2015: 109 · 2014: 70 · 2013: 55 · 2012: 36
- FWCI
- 55.84
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- GRADE, quality of evidence, indirectness, applicability, surrogate endpoints, head-to-head comparison, patient populations
- MeSH terms
- Clinical Competence, Humans, Review Literature as Topic, Meta-Analysis as Topic, Bias, Randomized Controlled Trials as Topic, Practice Guidelines as Topic, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
