GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Abstract
In the GRADE approach, randomized trials start as high-quality evidence and observational studies as low-quality evidence, but both can be rated down if most of the relevant evidence comes from studies that suffer from a high risk of bias. Well-established limitations of randomized trials include failure to conceal allocation, failure to blind, loss to follow-up, and failure to appropriately consider the intention-to-treat principle. More recently recognized limitations include stopping early for apparent benefit and selective reporting of outcomes according to the results. Key limitations of observational studies include use of inappropriate controls and failure to adequately adjust for prognostic imbalance. Risk of bias may vary across outcomes (e.g., loss to follow-up may be far less for all-cause mortality than for quality of life), a consideration that many systematic reviews ignore. In deciding whether to rate down for risk of bias--whether for randomized trials or observational studies--authors should not take an approach that averages across studies. Rather, for any individual outcome, when there are some studies with a high risk, and some with a low risk of bias, they should consider including only the studies with a lower risk of bias.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Certainty of evidence was assessed with GRADE tools for RCTs.
“Certainty of evidence was assessed with use of GRADE tools for RCTs (Guyatt et al., 2011; Ryan & Hill, 2016).”
From Effectiveness of Mental Health and Wellbeing Interventions for Children and Young People in Foster, Kinship, and Residential Care: Systematic Review and Meta-Analysis (2024) · cites this paper as Guyatt et al., 2011
The researchers will assess the evidence profiles for each identified outcome based on the GRADE approach.
“Additionally, the researchers (JL and WS) will assess the evidence profiles for each of the identified outcomes based on the GRADE (Grading of Recommendations, Assessment, Development and Evaluation) approach to reviewing evidence [86–88].”
From Acceptance and commitment therapy for chronic pain: Protocol of a systematic review and individual participant data meta-Analysis (2019) · cites this paper as [88]
Relevant criteria from the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) handbook were used to assess bias in all other study designs.
“Relevant criteria from the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) handbook were used to assess bias in all other study designs [37].”
From Computerized cognitive behavior therapy for anxiety and depression in rural areas:a systematic review (2015) · cites this paper as [37]
The overall quality of evidence was graded using the GRADE approach, considering study limitations, inconsistency, indirectness, imprecision and publication bias.
“The overall quality of evidence was graded using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach, considering study limitations, inconsistency, indirectness, imprecision and publication bias [40–44].”
Each included study was assessed for risk of potential bias following the GRADE approach.
“We also assessed each study for risk of potential bias following the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach and recorded the information in the extraction template469.”
From Health effects associated with exposure to secondhand smoke: a Burden of Proof study (2024) · cites this paper as [469]
Quality of evidence in included systematic reviews was assessed by outcome using the GRADE system criteria.
“Quality of evidence in included systematic reviews was assessed by outcome using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) system criteria [17-27].”
From Effects of Home Telemonitoring Interventions on Patients With Chronic Heart Failure: An Overview of Systematic Reviews (2015) · cites this paper as [20]
We applied the key criteria of the GRADE guidelines for observational studies to assess the methodological quality of nonrandomised studies.
“applied the key criteria of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidelines for observational studies to assess the methodological quality of nonrandomised studies.31”
Risk of bias assessment covered selection bias, performance bias, selective reporting bias, detection bias, attrition bias, and potential confounding.
“we assessed the risk of selection bias, performance bias, selective reporting bias, detection bias, attrition bias, and potential confounding [32].”
The overall quality of evidence for mental health indicators was assessed using the GRADE framework.
“The overall quality of research evidence for mental health indicators was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [32].”
The overall quality of evidence was rated as 'very low' according to the GRADE framework.
“The overall quality of evidence was rated as “very low” according to the GRADE framework [32].”
The review used a rigorous methodology following the PRISMA statement and GRADE framework.
“Second, the review used a rigorous methodology following the PRISMA statement for reporting systematic reviews and meta-analyses [26] and GRADE framework [32].”
RCTs are first categorised as high-quality evidence in the GRADE system, while observational studies are classified as low-quality data supporting estimates of intervention effects.
“RCTs are first categorised as high-quality evidence in the GRADE system, while observational studies are classified as low-quality data supporting estimates of intervention effects.”
The GRADE approach was used to determine the quality of evidence.
“To determine the quality of evidence, the GRADE approach was used [29–36].”
Risk of bias was assessed using the Cochrane risk-of-bias tool for RCTs, and certainty of the evidence was assessed using the GRADE approach.
“Risk of bias was assessed using the Cochrane risk‐of‐bias tool for RCTs, and certainty of the evidence was assessed using the GRADE approach.”
Cites
Prevention of Venous Thromboembolism
Geerts, W. H., Pineo, G. F., Heit, J. A., Bergqvist, D., Lassen, M. R., Colwell, C. W., & Ray, J. G. (2004) · CHEST Journal · doi:10.1378/chest.126.3_suppl.338s · Cited by 5,887 · see citing works
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · see citing works
Prevention of Venous Thromboembolism
Geerts, W. H., Heit, J. A., Clagett, G. P., Pineo, G. F., Colwell, C. W., Anderson, F. A., & Wheeler, H. B. (2001) · CHEST Journal · doi:10.1378/chest.119.1_suppl.132s · Cited by 4,368 · see citing works
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907 · see citing works
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · see citing works
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · see citing works
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · see citing works
Sanderson, S., Tatt, I. D., & Higgins, J. P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym018 · Cited by 1,531 · see citing works
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308 · see citing works
Users' guides to the medical literature : a manual for evidence-based clinical practice
Guyatt, G. H., Rennie, D., Meade, M. O., & Cook, D. J. (2002) · Cited by 1,196 · see citing works
Self-citationBowker, S. L., Majumdar, S. R., Veugelers, P., & Johnson, J. A. (2006) · Diabetes Care · doi:10.2337/diacare.29.02.06.dc05-1558 · Cited by 933 · see citing works
Greer, I. A., & Nelson-Piercy, C. (2005) · Blood · doi:10.1182/blood-2005-02-0626 · Cited by 797 · see citing works
Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu, S. (2009) · JAMA · doi:10.1001/jama.2009.1242 · Cited by 730 · see citing works
Methylprednisolone or naloxone treatment after acute spinal cord injury: 1-year follow-up data
Bracken, M. B., Shepard, M. J., Collins, W. F., Holford, T. R., Baskin, D. S., Eisenberg, H. M., Flamm, E. S., Leo-Summers, L. S., Maroon, J. C., Marshall, L. F., et al. (1992) · Journal of neurosurgery · doi:10.3171/jns.1992.76.1.0023 · Cited by 722 · see citing works
Bassler, D. (2010) · JAMA · doi:10.1001/jama.2010.310 · Cited by 613 · see citing works
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613 · see citing works
Safety of Low-Molecular-Weight Heparin in Pregnancy: A Systematic Review
Sanson, B., Lensing, A. W. A., Prins, M. H., Ginsberg, J. S., Баркаган, З. С., Lavenne-Pardonge, E., Brenner, B., Dulitzky, M., Nielsen, J., Boda, Z., et al. (1999) · Thrombosis and Haemostasis · doi:10.1055/s-0037-1614550 · Cited by 534 · see citing works
Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444 · see citing works
β-Blockers in Congestive Heart Failure: A Bayesian Meta-Analysis
Brophy, J. M., Joseph, L., & Rouleau, J. L. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-7-200104030-00008 · Cited by 389 · see citing works
Outcome Reporting in Industry-Sponsored Trials of Gabapentin for Off-Label Use
Vedula, S. S., Bero, L. A., Scherer, R. W., & Dickersin, K. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0906126 · Cited by 361 · see citing works
Pharmacological therapy of spinal cord injury during the acute phase
Pointillart, V., Petitjean, M. E., Wiart, L., Vital, J., Lassié, P., Thicoipe, M., & Dabadie, P. (2000) · Spinal Cord · doi:10.1038/sj.sc.3100962 · Cited by 316 · see citing works
Soares, H. P., Daniels, S., Kumar, A., Clarke, M. J., Scott, C., Swann, S., & Djulbegović, B. (2004) · BMJ · doi:10.1136/bmj.328.7430.22 · Cited by 310 · see citing works
Emerson, J. D., Burdick, E., Hoaglin, D. C., Mosteller, F. C., & Chalmers, T. C. (1990) · Controlled Clinical Trials · doi:10.1016/0197-2456(90)90175-2 · Cited by 241 · see citing works
Devereaux, P. J., Choi, P. T.-L., El-Dika, S., Bhandari, M., Montori, V. M., Schünemann, H. J., Garg, A. X., Busse, J. W., Heels-Ansdell, D., Ghali, W. A., et al. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.03.017 · Cited by 209 · see citing works
Self-citationBoloña, E. R., Uraga, M. V., Haddad, R. M., Tracz, M. J., Sideras, K., Kennedy, C. C., Caples, S. M., Erwin, P. J., & Montori, V. M. (2007) · Mayo Clinic Proceedings · doi:10.1016/s0025-6196(11)60963-4 · Cited by 184 · see citing works
Self-citationSurgical interventions for lumbar disc prolapse
Gibson, J. A., & Waddell, G. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001350.pub4 · Cited by 170 · see citing works
Meta-analysis of flavonoids for the treatment of haemorrhoids
Alonso‐Coello, P., Zhou, Q. X., Martínez‐Zapata, M. J., Mills, E. J., Heels‐Ansdell, D., Johanson, J. F., & Guyatt, G. H. (2006) · British journal of surgery · doi:10.1002/bjs.5378 · Cited by 168 · see citing works
Self-citationWhen (Not) to Stop a Clinical Trial for Benefit
Pocock, S. (2005) · JAMA · doi:10.1001/jama.294.17.2228 · Cited by 164 · see citing works
Practical problems in interim analyses, with particular regard to estimation
Pocock, S., & Hughes, M. D. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90059-7 · Cited by 160 · see citing works
Compression stockings for preventing deep vein thrombosis in airline passengers
Clarke, M. J., Broderick, C. A., Hopewell, S., Juszczak, E., & Eisinga, A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004002.pub3 · Cited by 133 · see citing works
Association Between Unreported Outcomes and Effect Size Estimates in Cochrane Meta-analyses
Furukawa, T. A., Watanabe, N., Omori, I. M., Montori, V. M., & Guyatt, G. H. (2007) · JAMA · doi:10.1001/jama.297.5.468-b · Cited by 115 · see citing works
Self-citationβ-blockers in congestive heart failure
Moskowitz, R. M., & Kukin, M. L. (1999) · Expert Opinion on Investigational Drugs · doi:10.1517/13543784.8.11.1795 · Cited by 95 · see citing works
Pitchforth, J., & Mengersen, K. L. (2012) · Wiley series in probability and statistics · doi:10.1002/9781118394472.ch7 · Cited by 74 · see citing works
Dobre, D., van Jaarsveld, C. H. M., DeJongste, M. J. L., Ruskamp, F. M. H., & Ranchor, A. V. (2006) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1234 · Cited by 54 · see citing works
McIntosh, H., & Jones, K. L. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000386.pub2 · Cited by 40 · see citing works
Randomized Trials Stopped Early for Benefit: A Systematic Review
Montori, V. M., Devereaux, P. J., Adhikari, N. K. J., Burns, K. E. A., Eggert, C. H., Briel, M., Lacchetti, C., Leung, T. W., Darling, E., Bryant, D. M., et al. (2005) · JAMA · doi:10.1001/jama.294.17.2203 · Cited by 26 · see citing works
Self-citationSinha, M. K., & Montori, V. M. (2006) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/14737167.6.5.603 · Cited by 14 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285 · see citing works
Self-citationAntithrombotic Therapy for VTE Disease
Kearon, C., Akl, E. A., Ornelas, J., Blaivas, A., Jimenez, D., Bounameaux, H., Huisman, M., King, C. S., Morris, T. A., Sood, N., et al. (2016) · Chest · doi:10.1016/j.chest.2015.11.026 · Cited by 4,830 · see citing works
Self-citationMcClave, S. A., Taylor, B. E., Martindale, R. G., Warren, M. M., Johnson, D. R., Braunschweig, C., McCarthy, M. S., Davanos, E., Rice, T. W., Cresci, G. A., et al. (2016) · Journal of Parenteral and Enteral Nutrition · doi:10.1177/0148607115621863 · Cited by 3,982 · see citing works
By the 2019 American Geriatrics Society Beers Criteria® Update Expert Panel (2019) · Journal of the American Geriatrics Society · doi:10.1111/jgs.15767 · Cited by 3,163 · see citing works
Japanese Society for Cancer of the Colon and Rectum, Watanabe, T., Muro, K., Ajioka, Y., Hashiguchi, Y., Ito, Y., Saito, Y., Hamaguchi, T., Ishida, H., Ishiguro, M., et al. (2018) · International Journal of Clinical Oncology · doi:10.1007/s10147-017-1101-6 · Cited by 2,467 · see citing works
GRADE guidelines: 7. Rating the quality of evidence—inconsistency
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Glasziou, P., Jaeschke, R., Akl, E. A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.017 · Cited by 2,320 · see citing works
Self-citationCannabinoids for Medical Use: A Systematic Review and Meta-analysis
Whiting, P. F., Wolff, R. F., Deshpande, S., Di Nisio, M., Duffy, S., Hernandez, A. V., Keurentjes, J. C., Lang, S., Misso, K., Ryder, S., et al. (2015) · JAMA · doi:10.1001/jama.2015.6358 · Cited by 2,169 · see citing works
CINeMA: an approach for assessing confidence in the results of a network meta-analysis
Nikolakopoulou, A., Higgins, J. P. T., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., & Salanti, G. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003082 · Cited by 1,935 · see citing works
Hashiguchi, Y., Muro, K., Saitō, Y., Ito, Y., Ajioka, Y., Hamaguchi, T., Hasegawa, K., Hotta, K., Ishida, H., Ishiguro, M., et al. (2019) · International Journal of Clinical Oncology · doi:10.1007/s10147-019-01485-z · Cited by 1,936 · see citing works
Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812 · see citing works
Self-citationThompson, A. J., Baranzini, S. E., Geurts, J. J. G., Hemmer, B., & Ciccarelli, O. (2018) · The Lancet · doi:10.1016/s0140-6736(18)30481-1 · Cited by 1,856 · see citing works
Breast Cancer Screening for Women at Average Risk
Oeffinger, K. C., Fontham, E. T. H., Etzioni, R., Herzig, A. H., Michaelson, J. S., Shih, Y. T., Walter, L. C., Church, T. R., Flowers, C. R., LaMonte, S. J., et al. (2015) · JAMA · doi:10.1001/jama.2015.12783 · Cited by 1,661 · see citing works
Kirkpatrick, A. W., Roberts, D. J., Waele, J. J. D., Jaeschke, R., Malbrain, M. L. N. G., Keulenaer, B. L. D., Duchesne, J. C., Björck, M., Leppäniemi, A. K., Ejike, J. C., et al. (2013) · Intensive Care Medicine · doi:10.1007/s00134-013-2906-z · Cited by 1,630 · see citing works
By the 2023 American Geriatrics Society Beers Criteria® Update Expert Panel (2023) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18372 · Cited by 1,597 · see citing works
Taylor, B. E., McClave, S. A., Martindale, R. G., Warren, M. M., Johnson, D. R., Braunschweig, C., McCarthy, M. S., Davanos, E., Rice, T. W., Cresci, G. A., et al. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000001525 · Cited by 1,476 · see citing works
Andrews, J., Guyatt, G., Oxman, A. D., Alderson, P., Dahm, P., Falck-Ytter, Y., Nasser, M., Meerpohl, J., Post, P. N., Kunz, R., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.03.013 · Cited by 1,416 · see citing works
Self-citationCarson, V., Hunter, S., Kuzik, N., Gray, C. E., Poitras, V. J., Chaput, J.-P., Saunders, T. J., Katzmarzyk, P. T., Okely, A. D., Connor Gorber, S., et al. (2016) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2015-0630 · Cited by 1,380 · see citing works
Evaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · see citing works
Andrews, J. C., Schünemann, H. J., Oxman, A. D., Pottie, K. C., Meerpohl, J. J., Alonso‐Coello, P., Rind, D. M., Montori, V. M., Brito, J. P., Norris, S. L., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.02.003 · Cited by 1,332 · see citing works
Self-citationLugli, A., Kirsch, R., Ajioka, Y., Bosman, F. T., Cathomas, G., Dawson, H., Zimaity, H. E., Fléjou, J., Hansen, T. P., Hartmann, A., et al. (2017) · Modern Pathology · doi:10.1038/modpathol.2017.46 · Cited by 1,174 · see citing works
Chaput, J.-P., Willumsen, J., Bull, F., Chou, R., Ekelund, U., Firth, J., Jago, R., Ortega, F. B., & Katzmarzyk, P. T. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-01037-z · Cited by 1,277 · see citing works
Drug treatments for covid-19: living systematic review and network meta-analysis
Siemieniuk, R. A., Bartoszko, J. J., Zeraatkar, D., Kum, E., Qasim, A., Díaz Martinez, J. P., Izcovich, A., Rochwerg, B., Lamontagne, F., Han, M. A., et al. (2020) · BMJ · doi:10.1136/bmj.m2980 · Cited by 846 · see citing works
Self-citationPsychological therapies for people with borderline personality disorder.
Stoffers-Winterling, J. M., Völlm, B. A., Rücker, G., Timmer, A., Huband, N., & Lieb, K. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005652.pub2 · Cited by 895 · see citing works
Sucrose for analgesia in newborn infants undergoing painful procedures
Stevens, B., Yamada, J., & Ohlsson, A. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001069.pub2 · Cited by 882 · see citing works
Determinants of patient satisfaction: a systematic review
Batbaatar, E., Dorjdagva, J., Luvsannyam, A., Savino, M. M., & Amenta, P. (2016) · Perspectives in Public Health · doi:10.1177/1757913916634136 · Cited by 872 · see citing works
GRADE guidelines: 12. Preparing Summary of Findings tables—binary outcomes
Guyatt, G. H., Oxman, A. D., Santesso, N., Helfand, M., Vist, G., Kunz, R., Brozek, J., Norris, S., Meerpohl, J., Djulbegovic, B., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.01.012 · Cited by 836 · see citing works
Self-citationStrunz, E. C., Addiss, D. G., Stocks, M. E., Ogden, S., Utzinger, J., & Freeman, M. C. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001620 · Cited by 807 · see citing works
Carson, V., Lee, E., Hewitt, L., Jennings, C. A., Hunter, S., Kuzik, N., Stearns, J. A., Unrau, S. P., Poitras, V. J., Gray, C. E., et al. (2017) · BMC Public Health · doi:10.1186/s12889-017-4860-0 · Cited by 791 · see citing works
GRADE guidelines: 13. Preparing Summary of Findings tables and evidence profiles—continuous outcomes
Guyatt, G. H., Thorlund, K., Oxman, A. D., Walter, S. D., Patrick, D., Furukawa, T. A., Johnston, B. C., Karanicolas, P., Akl, E. A., Vist, G., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.08.001 · Cited by 772 · see citing works
Self-citationGuyatt, G., Oxman, A. D., Sultan, S., Brozek, J., Glasziou, P., Alonso-Coello, P., Atkins, D., Kunz, R., Montori, V., Jaeschke, R., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.01.006 · Cited by 762 · see citing works
Self-citationSystematic review and meta-analysis of psychological therapies for children with chronic pain
Fisher, E., Heathcote, L., Palermo, T. M., de C Williams, A. C., Lau, J., & Eccleston, C. (2014) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsu008 · Cited by 710 · see citing works
Early palliative care for adults with advanced cancer
Haun, M. W., Estel, S., Rücker, G., Friederich, H.-C., Villalobos, M., Thomas, M., & Hartmann, M. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011129.pub2 · Cited by 757 · see citing works
Comparison of Weight Loss Among Named Diet Programs in Overweight and Obese Adults: A Meta-analysis
Johnston, B. C., Kanters, S., Bandayrel, K., Wu, P., Naji, F., Siemieniuk, R. A., Ball, G. D. C., Busse, J. W., Thorlund, K., Guyatt, G., et al. (2014) · JAMA · doi:10.1001/jama.2014.10397 · Cited by 704 · see citing works
Self-citationThe Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis
Liu, Q., Peng, W., Zhang, F., Hu, R., Li, Y., & Yan, W. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4807 · Cited by 691 · see citing works
Systematic review of physical activity and health in the early years (aged 0-4 years)
Timmons, B. W., LeBlanc, A. G., Carson, V., Connor Gorber, S., Dillman, C., Janssen, I., Kho, M. E., Spence, J. C., Stearns, J. A., & Tremblay, M. S. (2012) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/h2012-070 · Cited by 626 · see citing works
Selenium for preventing cancer
Vinceti, M., Filippini, T., Del Giovane, C., Dennert, G., Zwahlen, M., Brinkman, M., Zeegers, M. P., Horneber, M., D'Amico, R., & Crespi, C. M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005195.pub4 · Cited by 649 · see citing works
Meader, N., King, K., Llewellyn, A., Norman, G., Brown, J., Rodgers, M., Moe-Byrne, T., Higgins, J., Sowden, A., & Stewart, G. B. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-82 · Cited by 560 · see citing works
Pérez‐Escamilla, R., Martinez‐Brockman, J. L., & Segura‐Pérez, S. (2016) · Maternal and Child Nutrition · doi:10.1111/mcn.12294 · Cited by 597 · see citing works
Saunders, T. J., Gray, C. E., Poitras, V. J., Chaput, J.-P., Janssen, I., Katzmarzyk, P. T., Olds, T., Connor Gorber, S., Kho, M. E., Sampson, M., et al. (2016) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2015-0626 · Cited by 589 · see citing works
“And for indirectness, we considered all studies reporting the outcome, and examined whether studies included the populations, interventions, comparators, or outcomes of interest (Guyatt et al. 2011c).”
BackgroundMethodInfluentialWatanabe, T., Itabashi, M., Shimada, Y., Tanaka, S., Ito, Y., Ajioka, Y., Hamaguchi, T., Hyodo, I., Igarashi, M., Ishida, H., et al. (2015) · International Journal of Clinical Oncology · doi:10.1007/s10147-015-0801-z · Cited by 591 · see citing works
Ludyga, S., Gerber, M., Brand, S., Holsboer‐Trachsler, E., & Pühse, U. (2016) · Psychophysiology · doi:10.1111/psyp.12736 · Cited by 589 · see citing works
Fatima, Y., Doi, S. A. R., & Mamun, A. A. (2015) · Obesity Reviews · doi:10.1111/obr.12245 · Cited by 577 · see citing works
Davenport, M. H., Ruchat, S.-M., Poitras, V. J., Jaramillo Garcia, A., Gray, C. E., Barrowman, N., Skow, R. J., Meah, V. L., Riske, L., Sobierajski, F., et al. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099355 · Cited by 579 · see citing works
Machado, G. C., Maher, C. G., Ferreira, P. H., Pinheiro, M. B., Lin, C.-W. C., Day, R. O., McLachlan, A. J., & Ferreira, M. L. (2015) · BMJ · doi:10.1136/bmj.h1225 · Cited by 580 · see citing works
Abraha, I., Rimland, J. M., Trotta, F. M., Dell'Aquila, G., Cruz-Jentoft, A., Petrovic, M., Gudmundsson, A., Soiza, R., O'Mahony, D., Guaita, A., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-012759 · Cited by 557 · see citing works
Sucrose for analgesia in newborn infants undergoing painful procedures
Stevens, B., Yamada, J., Ohlsson, A., Haliburton, S., & Shorkey, A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001069.pub5 · Cited by 569 · see citing works
Chiavaroli, L., Viguiliouk, E., Nishi, S., Blanco Mejia, S., Rahelić, D., Kahleová, H., Salas-Salvadó, J., Kendall, C., & Sievenpiper, J. (2019) · Nutrients · doi:10.3390/nu11020338 · Cited by 548 · see citing works
Huguet, A., Hayden, J. A., Stinson, J., McGrath, P. J., Chambers, C. T., Tougas, M. E., & Wozney, L. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-71 · Cited by 531 · see citing works
Homocysteine lowering interventions for preventing cardiovascular events
Martí-Carvajal, A. J., Solà, I., Lathyris, D., & Salanti, G. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006612.pub2 · Cited by 520 · see citing works
Oral language deficits in familial dyslexia: A meta-analysis and review.
Snowling, M. J., & Melby‐Lervåg, M. (2016) · Psychological Bulletin · doi:10.1037/bul0000037 · Cited by 495 · see citing works
Polymyalgia Rheumatica and Giant Cell Arteritis
Buttgereit, F., Dejaco, C., Matteson, E. L., & Dasgupta, B. (2016) · JAMA · doi:10.1001/jama.2016.5444 · Cited by 492 · see citing works
Wesseloo, R., Kamperman, A. M., Munk-Olsen, T., Pop, V. J. M., Kushner, S. A., & Bergink, V. (2016) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2015.15010124 · Cited by 469 · see citing works
Beta-blockers for hypertension
Wiysonge, C. S., Bradley, H., Volmink, J. A., Mayosi, B. M., & Opie, L. H. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002003.pub5 · Cited by 475 · see citing works
Prevention of Type 2 Diabetes by Lifestyle Changes: A Systematic Review and Meta-Analysis
Uusitupa, M. I. J., Khan, T. A., Viguiliouk, E., Kahleová, H., Rivellese, A. A., Hermansen, K., Pfeiffer, A. F. H., Thanopoulou, A. C., Salas‐Salvadó, J., Schwab, U., et al. (2019) · Nutrients · doi:10.3390/nu11112611 · Cited by 488 · see citing works
Lemyre, B., Davis, P. G., Paoli, A. G. D., & Kirpalani, H. M. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003212.pub3 · Cited by 456 · see citing works
Use of the GRADE approach in systematic reviews and guidelines
Granholm, A., Alhazzani, W. A., & Møller, M. H. (2019) · British Journal of Anaesthesia · doi:10.1016/j.bja.2019.08.015 · Cited by 455 · see citing works
Psychological therapies for people with borderline personality disorder
Storebø, O. J., Stoffers-Winterling, J. M., Völlm, B. A., Kongerslev, M. T., Mattivi, J. T., Jørgensen, M. S., Faltinsen, E., Todorovac, A., Sales, C. P., Callesen, H. E., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012955.pub2 · Cited by 450 · see citing works
Johnson, P. I., Sutton, P., Atchley, D. S., Koustas, E., Lam, J., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307893 · Cited by 449 · see citing works
“Evaluation factors Summary of criteria Downgrading factors Risk of bias Study limitations include a substantial risk of bias across the body of evidence.”
BackgroundMethodAssessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions
Viswanathan, M., Ansari, M., Berkman, N., Chang, S. M., Hartling, L., McPheeters, M., Santaguida, P., Shamliyan, T., Singh, K., Tsertsvadze, A., et al. (2012) · doi:10.23970/ahrqepcmethguide2 · Cited by 731 · see citing works
Yu, J.-T., Xu, W., Tan, C.-C., Andrieu, S., Suckling, J., Evangelou, E., Pan, A., Zhang, C., Jia, J., Feng, L., et al. (2020) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp-2019-321913 · Cited by 482 · see citing works
Margulis, A. V., Pladevall, M., Riera‐Guàrdia, N., Varas‐Lorenzo, C., Hazell, L., Berkman, N. D., Viswanathan, M., & Perez‐Gutthann, S. (2014) · Clinical Epidemiology · doi:10.2147/clep.s66677 · Cited by 442 · see citing works
Krishnan, U., Mousa, H. M., Dall’Oglio, L., Homaira, N., Rosen, R., Faure, C. C., & Gottrand, F. (2016) · Journal of Pediatric Gastroenterology and Nutrition · doi:10.1097/mpg.0000000000001401 · Cited by 442 · see citing works
Auger, R. R., Burgess, H. J., Emens, J. S., Deriy, L. V., Thomas, S. M., & Sharkey, K. M. (2015) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.5100 · Cited by 437 · see citing works
Rooney, A. A., Boyles, A. L., Wolfe, M. S., Bucher, J. R., & Thayer, K. A. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307972 · Cited by 434 · see citing works
Pinheiro, M. B., Oliveira, J. S., Bauman, A. E., Fairhall, N., Kwok, W. S., & Sherrington, C. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-01040-4 · Cited by 435 · see citing works
Sampasa-Kanyinga, H., Colman, I., Goldfield, G. S., Janssen, I., Wang, J., Podinic, I., Tremblay, M. S., Saunders, T. J., Sampson, M., & Chaput, J.-P. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-00976-x · Cited by 430 · see citing works
Mueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400 · see citing works
Evidence-based clinical practice guidelines for Liver Cirrhosis 2020
Yoshiji, H., Nagoshi, S., Akahane, T., Asaoka, Y., Ueno, Y., Ogawa, K., Kawaguchi, T., Kurosaki, M., Sakaida, I., Shimizu, M., et al. (2021) · Journal of Gastroenterology · doi:10.1007/s00535-021-01788-x · Cited by 427 · see citing works
Romano, C., van Wynckel, M., Hulst, J. M., Broekaert, I. J., Bronský, J., Dall’Oglio, L., Mis, N. F., Hojsak, I., Orel, R., Papadopoulou, A., et al. (2017) · Journal of Pediatric Gastroenterology and Nutrition · doi:10.1097/mpg.0000000000001646 · Cited by 418 · see citing works
Japanese guidelines for the management of acute pancreatitis: Japanese Guidelines 2015
Yokoe, M., Takada, T., Mayumi, T., Yoshida, M., Isaji, S., Wada, K., Itoi, T., Sata, N., Gabata, T., Igarashi, H., et al. (2015) · Journal of Hepato-Biliary-Pancreatic Sciences · doi:10.1002/jhbp.259 · Cited by 416 · see citing works
Chaput, J., Gray, C. E., Poitras, V. J., Carson, V., Gruber, R., Birken, C. S., MacLean, J. E., Aubert, S., Sampson, M., & Tremblay, M. S. (2017) · BMC Public Health · doi:10.1186/s12889-017-4850-2 · Cited by 415 · see citing works
Sacco, S., Amin, F. M., Ashina, M., Bendtsen, L., Deligianni, C. I., Gil‐Gouveia, R., Katsarava, Z., MaassenVanDenBrink, A., Martelletti, P., Mitsikostas, D. D., et al. (2022) · The Journal of Headache and Pain · doi:10.1186/s10194-022-01431-x · Cited by 413 · see citing works
Lahti, M., Hätönen, H. M., & Valimaki, M. A. (2013) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.12.017 · Cited by 410 · see citing works
Weibel, S., Jelting, Y., Pace, N. L., Helf, A., Eberhart, L. H. J., Hahnenkamp, K., Hollmann, M. W., Poepping, D. M., Schnabel, A., & Kranke, P. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009642.pub3 · Cited by 417 · see citing works
Psychological interventions to foster resilience in healthcare professionals
Kunzler, A. M., Helmreich, I., Chmitorz, A., König, J., Binder, H., Wessa, M., & Lieb, K. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012527.pub2 · Cited by 390 · see citing works
ERS clinical practice guidelines: high-flow nasal cannula in acute respiratory failure
Oczkowski, S. J. W., Ergan, B., Bos, L. D. J., Chatwin, M., Ferrer, M., Gregoretti, C., Heunks, L. M. A., Frat, J., Longhini, F., Nava, S., et al. (2021) · European Respiratory Journal · doi:10.1183/13993003.01574-2021 · Cited by 401 · see citing works
Hughes, C. G., Boncyk, C. S., Culley, D. J., Fleisher, L. A., Leung, J. M., McDonagh, D. L., Gan, T. J., McEvoy, M. D., Miller, T. E., & Workgroup, F. T. P. Q. I. (. 6. (2020) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000004641 · Cited by 405 · see citing works
A systematic review of interventions for reducing parental vaccine refusal and vaccine hesitancy
Sadaf, A., Richards, J. L., Glanz, J. M., Salmon, D. A., & Omer, S. B. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.07.013 · Cited by 392 · see citing works
Understanding GRADE: an introduction
Goldet, G., & Howick, J. (2013) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12018 · Cited by 390 · see citing works
Johnson, J. A., Rash, J. A., Campbell, T. S., Savard, J., Gehrman, P. R., Perlis, M., Carlson, L. E., & Garland, S. N. (2016) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2015.07.001 · Cited by 371 · see citing works
Akiyama, S., Hamdeh, S., Micić, D., & Sakuraba, A. (2020) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2020-218946 · Cited by 383 · see citing works
Mustafa, R. A., Santesso, N., Brozek, J., Akl, E. A., Walter, S. D., Norman, G., Kulasegaram, M., Christensen, R., Guyatt, G. H., Falck-Ytter, Y., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.02.004 · Cited by 370 · see citing works
Self-citationSacco, S., Bendtsen, L., Ashina, M., Reuter, U., Terwindt, G. M., Mitsikostas, D. D., & Martelletti, P. (2019) · The Journal of Headache and Pain · doi:10.1186/s10194-018-0955-y · Cited by 371 · see citing works
Medication Therapy Management Interventions in Outpatient Settings
Viswanathan, M., Kahwati, L. C., Golin, C. E., Blalock, S. J., Coker‐Schwimmer, E., Posey, R., & Lohr, K. N. (2014) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2014.5841 · Cited by 363 · see citing works
Obesity as a risk factor for triple-negative breast cancers: a systematic review and meta-analysis
Pierobon, M., & Frankenfeld, C. L. (2012) · Breast Cancer Research and Treatment · doi:10.1007/s10549-012-2339-3 · Cited by 357 · see citing works
Kitsiou, S., Paré, G., & Jaana, M. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4174 · Cited by 327 · see citing works
Arnberg, F. K., Linton, S. J., Hultcrantz, M., Heintz, E., & Jonsson, U. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098118 · Cited by 330 · see citing works
Koustas, E., Lam, J., Sutton, P., Johnson, P. I., Atchley, D. S., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307177 · Cited by 349 · see citing works
Burton, R., Henn, C., Lavoie, D., O'Connor, R., Perkins, C., Sweeney, K., Greaves, F., Ferguson, B., Beynon, C. M., Belloni, A., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)32420-5 · Cited by 346 · see citing works
Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)
Storebø, O. J., Ramstad, E., Krogh, H. B., Nilausen, T. D., Skoog, M., Holmskov, M., Rosendal, S., Groth, C., Magnusson, F. L., Moreira-Maia, C. R., et al. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009885.pub2 · Cited by 347 · see citing works
Systematic review of sedentary behaviour and health indicators in the early years (aged 0–4 years)
LeBlanc, A. G., Spence, J. C., Carson, V., Gorber, S. C., Dillman, C. J., Janssen, I., Kho, M. E., Stearns, J. A., Timmons, B. W., & Tremblay, M. S. (2012) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/h2012-063 · Cited by 340 · see citing works
Nanda, K. S., Cheifetz, A. S., & Moss, A. C. (2012) · The American Journal of Gastroenterology · doi:10.1038/ajg.2012.363 · Cited by 336 · see citing works
Brożek, J., Canelo‐Aybar, C., Akl, E. A., Bowen, J. M., Bucher, J. R., Chiu, W. A., Cronin, M., Djulbegović, B., Falavigna, M., Guyatt, G. H., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.018 · Cited by 342 · see citing works
Self-citationToric Intraocular Lenses in the Correction of Astigmatism During Cataract Surgery
Kessel, L., Andresen, J. L., Tendal, B., Erngaard, D., Flesner, P., & Hjortdal, J. (2015) · Ophthalmology · doi:10.1016/j.ophtha.2015.10.002 · Cited by 333 · see citing works
Han, S., Woo, S., Kim, Y. J., & Suh, C. H. (2018) · European Urology · doi:10.1016/j.eururo.2018.03.030 · Cited by 332 · see citing works
A.S.P.E.N. Clinical Guidelines
Boullata, J. I., Gilbert, K., Sacks, G. S., Labossiere, R., Crill, C., Goday, P. S., Kumpf, V. J., Mattox, T. W., Plogsted, S. W., Holcombe, B. J., et al. (2014) · Journal of Parenteral and Enteral Nutrition · doi:10.1177/0148607114521833 · Cited by 321 · see citing works
Subramaniam, P., Ho, J. J., & Davis, P. G. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001243.pub3 · Cited by 317 · see citing works
Hinkelbein, J., Lamperti, M., Åkeson, J., Santos, J., Costa, J. N., ROBERTIS, E. D., Longrois, D., Novak–Jankovič, V., Petrini, F., Struys, M. M. R. F., et al. (2017) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000000683 · Cited by 315 · see citing works
Memtsoudis, S. G., Cozowicz, C., Bekeris, J., Bekere, D., Liu, J., Soffin, E. M., Mariano, E. R., Johnson, R. L., Hargett, M. J., Lee, B. H., et al. (2019) · British Journal of Anaesthesia · doi:10.1016/j.bja.2019.05.042 · Cited by 314 · see citing works
Acute myocardial infarction and influenza: a meta-analysis of case–control studies
Barnes, M., Heywood, A. E., Mahimbo, A., Rahman, B., Newall, A. T., & MacIntyre, C. R. (2015) · Heart · doi:10.1136/heartjnl-2015-307691 · Cited by 311 · see citing works
- Authors
- Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y.
- Affiliations
- McMaster University (CA)
- Nasjonalt Kunnskapssenter for Helsetjenesten (NO)
- University of Basel (CH)
- University at Buffalo, State University of New York (US)
- Case Western Reserve University (US)
- Corresponding author
- Guyatt, Gordon H.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2011-04
- Volume
- 64
- Issue
- 4
- Pages
- 407-415
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W1981680557
- Cited by
- 2,798
- References
- 37
- Citations by year
- 2026: 130 · 2025: 182 · 2024: 238 · 2023: 199 · 2022: 264 · 2021: 280 · 2020: 284 · 2019: 198 · 2018: 184 · 2017: 180 · 2016: 176 · 2015: 179 · 2014: 120 · 2013: 95 · 2012: 65
- FWCI
- 100.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- GRADE, risk of bias, quality of evidence, observational studies, allocation concealment, blinding, loss to follow-up, intention-to-treat analysis, early stopping, selective reporting
- MeSH terms
- Female, Humans, Male, Quality Assurance, Health Care, Randomized Controlled Trials as Topic, Practice Guidelines as Topic, Publication Bias, Risk Assessment, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
