Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients
Open accessAbstract
Main concepts : The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach defines quality of evidence as confidence in effect estimates; this conceptualization can readily be applied to bodies of evidence estimating the risk of future of events (that is, prognosis) in broadly defined populations In the field of prognosis, a body of observational evidence (including single arms of randomized controlled trials) begins as high quality evidence. The five domains GRADE considers in rating down confidence in estimates of treatment effect-that is, risk of bias, imprecision, inconsistency, indirectness, and publication bias-as well as the GRADE criteria for rating up quality, also apply to estimates of the risk of future of events from a body of prognostic studies Applying these concepts to systematic reviews of prognostic studies provides a ful approach to determine confidence in estimates of overall prognosis in broad populations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These factors might account for some of the residual statistical heterogeneity seen for some outcomes, albeit I2 is commonly inflated in meta-analyses of observational data, and nevertheless the effects seen were large and probably clinically important in all adjusted studies.
“These factors might account for some of the residual statistical heterogeneity seen for some outcomes, albeit I2 is commonly inflated in meta-analyses of observational data,21, 22 and nevertheless the effects seen were large and probably clinically important in all adjusted studies.”
While there is no formal guidance for GRADE in systematic reviews of prevalence, there is some guidance into the use of GRADE for baseline risk or overall prognosis which may be useful for these types of reviews.
“While there is no formal guidance for GRADE in systematic reviews of prevalence, there is some guidance into the use of GRADE for baseline risk or overall prognosis [25] which may be useful for these types of reviews.”
From How are systematic reviews of prevalence conducted? A methodological study (2020) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305 · see citing works
Self-citationOperating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · see citing works
GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094 · see citing works
Self-citationArcelus, J., Mitchell, A. J., Wales, J., & Nielsen, S. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.74 · Cited by 2,864 · see citing works
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508 · see citing works
Self-citationPrognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research
Riley, R. D., Hayden, J. A., Steyerberg, E. W., Moons, K. G. M., Abrams, K., Kyzas, P. A., Malats, N., Briggs, A., Schroter, S., Altman, D. G., et al. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001380 · Cited by 823 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 22
- Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, et al, for the PROGRESS Group. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ 2013;346:e5595.
- Hingorani A, van der Windt D, Riley R, Abrams K, Moons KG, Steyerberg EW, et al, for the PROGRESS Group. Prognosis research strategy (PROGRESS) 4: stratified medicine research. BMJ 2013;345:e5793.
- Thompson M, Vodicka TA, Blair PS, Buckley DI, Heneghan C, Hay AD, for the TARGET Programme Team. Duration of symptoms of respiratory tract infections in children: systematic review. BMJ 2013;347:f7027.
- Riley RD, Steyerberg EW. Meta-analysis of a binary outcome using individual participant data and aggregate data. Res Synth Methods 2010;1:2-19.
- Hamza TH, van Houwelingen HC, Stijnen T. The binomial distribution of meta-analysis was preferred to model within-study variability. J Clin Epidemiol 2008;61:41-51.
- Spencer FA, Iorio A, You J, Murad MH, Schünemann HJ, Vandvik PO, et al. Uncertainties in baseline risk estimates and confidence in treatment effects. BMJ 2012;345:e7401.
- Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, et al. GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. J Clin Epidemiol 2013;66:158-72.
- Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, et al. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. J Clin Epidemiol 2013;66:173-83.
- Lopes LC, Spencer F a, Neumann I, Ventresca M, Ebrahim S, Zhou Q, et al. Bleeding risk in atrial fibrillation patients taking vitamin K antagonists: systematic review and meta-analysis. Clin Pharmacol Ther 2013;94:367-75.
- Hayden J, van der Windt D, Cartwright JL, Côté P, Bombardier C. Assessing bias in studies of prognostic factors. Ann Intern Med 2013;158:280-6.
- Mohan KM, Wolfe CD, Rudd AG, Heuschmann PU, Kolominsky-Rabas PL, Grieve AP. Risk and cumulative risk of stroke recurrence: a systematic review and meta-analysis. Stroke 2011;42:1489-94.
- Yousef F, Cardwell C, Cantwell MM, Galway K, Johnston BT, Murray L. The incidence of esophageal cancer and high-grade dysplasia in Barrett's esophagus: a systematic review and meta-analysis. Am J Epidemiol 2008;168:237-49.
- Januel J-M, Chen G, Riffieux C, Quan H, Douketis JD, Crowther MA, et al, for the IMECCHI Group. Symptomatic in-hospital deep vein thrombosis and pulmonary embolism following hip and knee arthroplasty among patients receiving recommended prophylaxis: a systematic review. JAMA 2012;307:294-303.
- Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al, for the GRADE Working Group. GRADE guidelines: 7. Rating the quality of evidence-inconsistency. J Clin Epidemiol 2011;64:1294-302.
- Rücker G, Schwarzer G, Carpenter JR, Schumacher M. Undue reliance on I(2) in assessing heterogeneity may mislead. BMC Med Res Methodol 2008;8:79.
- Sun X, Briel M, Walter SD, Guyatt GH. Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ 2010;340:c117.
- Su Y, Norris JL, Zang C, Peng Z, Wang N. Incidence of hepatitis C virus infection in patients on hemodialysis: a systematic review and meta-analysis. Hemodial Int 2013;17:532-41.
- Hemingway H, Riley RD, Altman DG. Ten steps towards improving prognosis research. BMJ 2009;339:b4184.
- Bukh J, Wantzin P, Krogsgaard K, Knudsen F, Purcell RH, Miller RH. High prevalence of hepatitis C virus (HCV) RNA in dialysis patients: failure of commercially available antibody tests to identify a significant number of patients with HCV infection. J Infect Dis 1993;168:1343-8.
- Couroucé AM, Le Marrec N, Girault A, Ducamp S, Simon N. Anti-hepatitis C virus (anti-HCV) seroconversion in patients undergoing hemodialysis: comparison of second-and third- generation anti-HCV assays. Transfusion 1994;34:790-5.
- Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315:629-34.
- Oldenburg J, Pavlova A. Genetic risk factors for inhibitors to factors VIII and IX. Haemophilia 2006;12:15-22.
Cited by
Sorted by citations
Chu, D. K., Akl, E. A., Duda, S., Solo, K., Yaacoub, S., Schünemann, H. J., Chu, D. K., Akl, E. A., El-harakeh, A., Bognanni, A., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)31142-9 · Cited by 3,843 · see citing works
Self-citationTaylor, R. S., Taylor, R. J., Bayliss, S., Hagström, H., Nasr, P., Schattenberg, J. M., Ishigami, M., Toyoda, H., Wai-Sun Wong, V., Peleg, N., et al. (2020) · Gastroenterology · doi:10.1053/j.gastro.2020.01.043 · Cited by 1,069 · see citing works
Schünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911 · see citing works
Self-citationCompleting ‘Summary of findings’ tables and grading the certainty of the evidence
Schünemann, H. J., Higgins, J. P., Vist, G. E., Glasziou, P., Akl, E. A., Skoetz, N., Guyatt, G. H., & on behalf of the Cochrane GRADEing Methods Group (formerly Applicability and Recommendations Methods Group) and the Cochrane Statistical Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch14 · Cited by 669 · see citing works
Self-citationA guide to systematic review and meta-analysis of prediction model performance
Debray, T. P. A., Damen, J. A. A. G., Snell, K. I. E., Ensor, J., Hooft, L., Reitsma, J. B., Riley, R. D., & Moons, K. G. M. (2017) · BMJ · doi:10.1136/bmj.i6460 · Cited by 657 · see citing works
Foroutan, F., Guyatt, G., Zuk, V., Vandvik, P. O., Alba, A. C., Mustafa, R., Vernooij, R., Arevalo-Rodriguez, I., Munn, Z., Roshanov, P., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.12.023 · Cited by 465 · see citing works
Self-citationHow are systematic reviews of prevalence conducted? A methodological study
on behalf of the Prevalence Estimates Reviews – Systematic Review Methodology Group (PERSyst), Borges Migliavaca, C., Stein, C., Colpani, V., Barker, T. H., Munn, Z., & Falavigna, M. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00975-3 · Cited by 359 · see citing works
Self-citationWang, L., Guyatt, G. H., Kennedy, S. A., Romerosa, B., Kwon, H. Y., Kaushal, A., Chang, Y., Craigie, S., de Almeida, C. P. B., Couban, R. J., et al. (2016) · Canadian Medical Association Journal · doi:10.1503/cmaj.151276 · Cited by 334 · see citing works
Self-citationRighy, C., Rosa, R. G., da Silva, R. T. A., Kochhann, R., Migliavaca, C. B., Robinson, C. C., Teche, S. P., Teixeira, C., Bozza, F. A., & Falavigna, M. (2019) · Critical Care · doi:10.1186/s13054-019-2489-3 · Cited by 319 · see citing works
Self-citationUsing systematic reviews in guideline development: The GRADE approach
Zhang, Y., Akl, E. A., & Schünemann, H. J. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1313 · Cited by 260 · see citing works
Self-citationHospitalisation rates and predictors in people with dementia: a systematic review and meta-analysis
Shepherd, H., Livingston, G., Chan, J., & Sommerlad, A. (2019) · BMC Medicine · doi:10.1186/s12916-019-1369-7 · Cited by 232 · see citing works
“We then used the Cochrane Collaboration’s Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach to rate the confidence in the estimates given [14].”
MethodWang, L., Cohen, J. C., Devasenapathy, N., Hong, B. Y., Kheyson, S., Lu, D., Oparin, Y., Kennedy, S. A., Romerosa, B., Arora, N., et al. (2020) · British Journal of Anaesthesia · doi:10.1016/j.bja.2020.04.088 · Cited by 142 · see citing works
Aerobic Exercise and Weight Loss in Adults
Jayedi, A., Soltani, S., Emadi, A., Zargar, M.-S., & Najafi, A. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.52185 · Cited by 132 · see citing works
Welch, V. A., Akl, E. A., Pottie, K., Ansari, M. T., Briel, M., Christensen, R., Dans, A., Dans, L., Eslava-Schmalbach, J., Guyatt, G., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.01.015 · Cited by 88 · see citing works
Self-citationPredictors of gastrointestinal bleeding in adult ICU patients: a systematic review and meta-analysis
the GUIDE Group, Granholm, A., Zeng, L., Dionne, J. C., Perner, A., Marker, S., Krag, M., MacLaren, R., Ye, Z., Møller, M. H., et al. (2019) · Intensive Care Medicine · doi:10.1007/s00134-019-05751-6 · Cited by 83 · see citing works
Trivedi, P. J., Corpechot, C., Pares, A., & Hirschfield, G. M. (2016) · Hepatology · doi:10.1002/hep.28128 · Cited by 71 · see citing works
Geyskens, L., Jeuris, A., Deschodt, M., Van Grootven, B., Gielen, E., & Flamaing, J. (2022) · Age and Ageing · doi:10.1093/ageing/afac007 · Cited by 70 · see citing works
- Authors
- Iorio, A., Spencer, F. A., Falavigna, M., Alba, C., Lang, E., Burnand, B., McGinn, T., Hayden, J., Williams, K., Shea, B., Wolff, R., Kujpers, T., Perel, P., Vandvik, P. O., Glasziou, P., Schunemann, H., & Guyatt, G.
- Affiliations
- McMaster University (CA)
- Universidade Federal do Rio Grande do Sul (BR)
- Alberta Health Services (CA)
- Centre Hospitalier Universitaire Vaudois (CH)
- Northwell Health (US)
- Dalhousie University (CA)
- Royal Children's Hospital (AU)
- University of Ottawa (CA)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Dutch College of General Practitioners (NL)
- London School of Hygiene & Tropical Medicine (GB)
- Innlandet Hospital Trust (NO)
- Bond University (AU)
- Corresponding author
- Iorio, A.
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2015-03-16
- Volume
- 350
- Issue
- mar16 7
- Pages
- h870-h870
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2059333369
- Cited by
- 880
- References
- 7
- Citations by year
- 2026: 101 · 2025: 116 · 2024: 119 · 2023: 101 · 2022: 113 · 2021: 102 · 2020: 97 · 2019: 54 · 2018: 30 · 2017: 26 · 2016: 15 · 2015: 6
- FWCI
- 23.41
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- e-publications@bond (Bond University) (submittedVersion)
- Bond University Research Portal (Bond University) (submittedVersion, other-oa)
- IRIS (submittedVersion)
- SERVAL (Université de Lausanne) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- GRADE, quality of evidence, prognosis, risk of bias, imprecision, inconsistency, indirectness, publication bias, observational evidence
- MeSH terms
- Diagnosis, Humans, Prognosis, Review Literature as Topic, Risk, Sensitivity and Specificity, Bias, Practice Guidelines as Topic, Publication Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
