CoChat
Sign Up
Cited by 376Clinical practice guidelines implementationHealth Policy Implementation ScienceDelphi Technique in Research

The GRADE Evidence to Decision (EtD) framework for health system and public health decisions

Open access
for the GRADE Working Group, Moberg, Jenny, Oxman, Andrew D., Rosenbaum, Sarah, Schünemann, Holger J., Guyatt, Gordon, Flottorp, Signe, Glenton, Claire +4 more·2018·Health Research Policy and Systems
DOI 10.1186/s12961-018-0320-2PMC PMC5975536
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: To describe a framework for people making and using evidence-informed health system and public health recommendations and decisions. BACKGROUND: We developed the GRADE Evidence to Decision (EtD) framework for health system and public health decisions as part of the DECIDE project, in which we simultaneously developed frameworks for these and other types of healthcare decisions, including clinical recommendations, coverage decisions and decisions about diagnostic tests. DEVELOPING THE FRAMEWORK: Building on GRADE EtD tables, we used an iterative approach, including brainstorming, consultation of the literature and with stakeholders, and an international survey of policy-makers. We applied the framework to diverse examples, conducted workshops and user testing with health system and public health guideline developers and policy-makers, and observed and tested its use in real-life guideline panels. FINDINGS: All the GRADE EtD frameworks share the same basic structure, including sections for formulating the question, making an assessment and drawing conclusions. Criteria listed in the assessment section of the health system and public health framework cover the important factors for making these types of decisions; in addition to the effects and economic impact of an option, the priority of the problem, the impact of the option on equity, and its acceptability and feasibility are important considerations that can inform both whether and how to implement an option. Because health system and public health interventions are often complex, detailed implementation considerations should be made when making a decision. The certainty of the evidence is often low or very low, but decision-makers must still act. Monitoring and evaluation are therefore often important considerations for these types of decisions. We illustrate the different components of the EtD framework for health system and public health decisions by presenting their application in a framework adapted from a real-life guideline. DISCUSSION: This framework provides a structured and transparent approach to support policy-making informed by the best available research evidence, while making the basis for decisions accessible to those whom they will affect. The health system and public health EtD framework can also be used to facilitate dissemination of recommendations and enable decision-makers to adopt, and adapt, recommendations or decisions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12961-018-0320-2

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 44

  1. [1]Ciapponi A Lewin S Herrera CA Opiyo N Pantoja T Paulsen E Delivery arrangements for health systems in low-income countries: an overview of systematic reviews Cochrane Database Syst Rev 2017 9 CD011083 28901005 PMC5621087
  2. [2]Wiysonge CS Paulsen E Lewin S Ciapponi A Herrera CA Opiyo N Financial arrangements for health systems in low-income countries: an overview of systematic reviews Cochrane Database Syst Rev 2017 9 CD011084 28891235 PMC5618470
  3. [3]Herrera CA Lewin S Paulsen E Ciapponi A Opiyo N Pantoja T Governance arrangements for health systems in low-income countries: an overview of systematic reviews Cochrane Database Syst Rev 2017 9 CD011085 28895125 PMC5618451
  4. [4]Pantoja T Opiyo N Lewin S Paulsen E Ciapponi A Wiysonge CS Implementation strategies for health systems in low-income countries: an overview of systematic reviews Cochrane Database Syst Rev 2017 9 CD011086 28895659 PMC5621088
  5. [5]Guyatt GH Oxman AD Kunz R Falck-Ytter Y Vist GE Liberati A Going from evidence to recommendations BMJ 2008 336 1049 1051 10.1136/bmj.39493.646875.AE 18467413 PMC2376019 · DOI · Paper page
  6. [6]Andrews J Guyatt GH Oxman AD Alderson P Dahm P Falck-Ytter Y GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations J Clin Epidemiol 2013 66 719 725 10.1016/j.jclinepi.2012.03.013 23312392 · DOI · Paper page
  7. [7]Andrews J Schünemann HJ Oxman AD Kunz R Brozek J Vist G GRADE guidelines: 15. Going from evidence to recommendations: determinants of a recommendations direction and strength J Clin Epidemiol 2013 66 726 735 10.1016/j.jclinepi.2013.02.003 23570745 · DOI
  8. [8]Akl EA Kennedy C Konda K Caceres CF Horvath T Ayala G Using GRADE methodology for the development of public health guidelines for the prevention and treatment of HIV and other STIs among men who have sex with men and transgender people BMC Public Health 2012 12 386 10.1186/1471-2458-12-386 22640260 PMC3490932 · DOI
  9. [9]Treweek S Oxman AD Alderson P Bossuyt P Brożek J Davoli M Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE): protocol and preliminary results Implementation Sci 2013 8 6 10.1186/1748-5908-8-6 PMC3553065 23302501 · DOI
  10. [10]Alonso-Coello P Schünemann HJ Moberg J Brignardello-Petersen R Akl EA Davoli M GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction BMJ 2016 353 i2016 10.1136/bmj.i2016 27353417 · DOI · Paper page
  11. [12]Schünemann HJ Mustafa R Brozek J Santesso N Alonso-Coello P Guyatt G GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health J Clin Epidemiol 2016 76 89 98 10.1016/j.jclinepi.2016.01.032 26931285 · DOI · Paper page
  12. [13]Parmelli E Amato L Oxman AD Alonso-Coello P Brunetti M Moberg J GRADE Evidence to Decision (EtD) framework for coverage decisions Int J Technol Assess Health Care 2017 2017 33 2 28655365
  13. [14]Rosenbaum SE, Moberg J, Glenton C, Schünemann HJ, Lewin S, Akl E, et al. Developing Evidence to Decision frameworks and an interactive Evidence to Decision tool for making and using decisions and recommendations in health care. Global Challenges. 2018. 10.1002/gch2.201700081. PMC6607226 31565348 · DOI
  14. [15]Lavis JN Permanand G Oxman AD Lewin S Fretheim A SUPPORT tools for evidence-informed health policymaking (STP). 13. Preparing and using policy briefs to support evidence-informed policymaking Health Res Policy Syst 2009 7 Suppl 1 S13 10.1186/1478-4505-7-S1-S13 20018103 PMC3271824 · DOI
  15. [17]Vogel JP Oxman AD Glenton C Rosenbaum S Lewin S Gülmezoglu AM Policymakers’ and other stakeholders’ perceptions of key considerations for health system decisions and the presentation of evidence to inform those considerations: An international survey Health Res Policy Syst 2013 11 19 10.1186/1478-4505-11-19 23705832 PMC3672010 · DOI
  16. [18]World Health Organization. WHO recommendations on antenatal care for a positive pregnancy experience. Geneva: World Health Organization; 2016. http://www.who.int/reproductivehealth/publications/maternal_perinatal_health/anc-positive-pregnancy-experience/en/. Accessed 15 May 2018. 28079998
  17. [19]World Health Organization. WHO recommendation on community mobilization through facilitated participatory learning and action cycles with women’s groups for maternal and newborn health. Geneva: World Health Organization; 2014. http://apps.who.int/iris/bitstream/10665/127939/1/9789241507271_eng.pdf?ua=1. Accessed 15 May 2018. 25165805
  18. [20]Glenton C Lewin S Gülmezoglu AM Expanding the evidence base for global recommendations on health systems: strengths and challenges of the OptimizeMNH guidance process Implement Sci 2016 11 98 10.1186/s13012-016-0470-y 27430879 PMC4950654 · DOI
  19. [21]World Health Organization. Optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. Geneva: World Health Organization; 2012. http://www.optimizemnh.org. Accessed 15 May 2018. 23844452
  20. [22]World Health Organization. Health worker roles in providing safe abortion care and post-abortion contraception. Geneva: World Health Organization; 2015. http://www.who.int/reproductivehealth/publications/unsafe_abortion/abortion-task-shifting/en/. Accessed 15 May 2018. 26401543
  21. [23]Guldbrandsson K Stenström N Winzer R The DECIDE evidence to recommendation framework adapted to the public health field in Sweden Health Promot Int 2016 31 749 754 10.1093/heapro/dav060 26082448 · DOI
  22. [25]Choudhry NK Stelfox HT Detsky AS Relationships between authors of clinical practice guidelines and the pharmaceutical industry JAMA 2002 287 612 617 10.1001/jama.287.5.612 11829700 · DOI
  23. [26]Schunemann HJ Osborne M Moss J Manthous C Wagner G Sicilian L An official American Thoracic Society Policy statement: managing conflict of interest in professional societies Am J Respir Crit Care Med 2009 180 564 580 10.1164/rccm.200901-0126ST 19734351 · DOI
  24. [27]Boyd EA Bero LA Improving the use of research evidence in guideline development: 4. Managing conflicts of interests Health Res Policy Syst 2006 4 16 10.1186/1478-4505-4-16 17140441 PMC1693552 · DOI
  25. [28]Guyatt G Akl EA Hirsh J Kearon C Crowther M Gutterman D The vexing problem of guidelines and conflict of interest: a potential solution Ann Intern Med 2010 152 738 741 10.7326/0003-4819-152-11-201006010-00254 20479011 · DOI
  26. [31]Blencowe H Cousens S Bianchi Jassir F Chou D Mathers C National, regional, and worldwide estimates of stillbirth rates in 2015, with trends from 2000: a systematic analysis Lancet Glob Health 2016 4 e98 e108 10.1016/S2214-109X(15)00275-2 26795602 · DOI · Paper page
  27. [32]Fisk NM McKee M Atun R Relative and absolute addressability of global disease burden in maternal and perinatal health by investment in R&D Tropical Med Int Health 2011 16 662 668 10.1111/j.1365-3156.2011.02778.x 21470349 · DOI
  28. [33]Downe S Finlayson K Tunçalp Ӧ Metin Gülmezoglu A What matters to women: a systematic scoping review to identify the processes and outcomes of antenatal care provision that are important to healthy pregnant women BJOG 2016 123 529 539 10.1111/1471-0528.13819 26701735 · DOI
  29. [34]Oxman AD Fretheim A Lavis JN Lewin S SUPPORT Tools for evidence-informed health Policymaking (STP). 12. Finding and using research evidence about resource use and costs Health Res Policy Syst 2009 7 Suppl 1 S12 10.1186/1478-4505-7-S1-S12 20018102 PMC3271823 · DOI
  30. [35]Oxman AD Lavis JN Fretheim A Lewin S SUPPORT Tools for evidence-informed health Policymaking (STP). 17. Dealing with insufficient research evidence Health Res Policy Syst 2009 7 Suppl 1 S17 10.1186/1478-4505-7-S1-S17 20018107 PMC3271827 · DOI
  31. [36]Mangham-Jefferies L Pitt C Cousens S Mills A Schellenberg J Cost-effectiveness of strategies to improve the utilization and provision of maternal and newborn health care in low-income and lower-middle-income countries: a systematic review BMC Pregnancy Childbirth 2014 14 243 10.1186/1471-2393-14-243 25052536 PMC4223592 · DOI
  32. [37]Prost A Colbourn T Seward N Azad K Coomarasamy A Copas A Women's groups practising participatory learning and action to improve maternal and newborn health in low-resource settings: a systematic review and meta-analysis Lancet 2013 381 1736 1746 10.1016/S0140-6736(13)60685-6 23683640 PMC3797417 · DOI
  33. [38]Oxman AD Lavis JN Lewin S Fretheim A SUPPORT Tools for evidence-informed health Policymaking (STP) 10: Taking equity into consideration when assessing the findings of a systematic review Health Res Policy Syst 2009 7 Suppl 1 S10 10.1186/1478-4505-7-S1-S10 20018100 PMC3271821 · DOI
  34. [39]O'Neill J Tabish H Welch V Petticrew M Pottie K Clarke M Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health J Clin Epidemiol 2014 67 56 64 10.1016/j.jclinepi.2013.08.005 24189091 · DOI · Paper page
  35. [41]Houweling TA Tripathy P Nair N Rath S Rath S Gope R The equity impact of participatory women's groups to reduce neonatal mortality in India: secondary analysis of a cluster-randomised trial Int J Epidemiol 2013 42 520 532 10.1093/ije/dyt012 23509239 PMC3619953 · DOI
  36. [43]Colvin CJ de Heer J Winterton L Mellenkamp M Glenton C Noyes J A systematic review of qualitative evidence on barriers and facilitators to the implementation of task-shifting in midwifery services Midwifery 2013 29 1211 1221 10.1016/j.midw.2013.05.001 23769757 · DOI · Paper page
  37. [44]Schad M John J Towards a social discount rate for the economic evaluation of health technologies in Germany: an exploratory analysis Eur J Health Econ 2012 13 127 144 10.1007/s10198-010-0292-9 21170731 · DOI
  38. [45]Beauchamp TL Childress JF Principles of Biomedical Ethics 2009 Oxford Oxford University Press
  39. [46]Saarni SI Hofmann B Lampe K Lühmann D Mäkelä M Velasco-Garrido M Ethical analysis to improve decision-making on health technologies Bull World Health Organ 2008 86 617 623 10.2471/BLT.08.051078 18797620 PMC2649462 · DOI
  40. [47]Downe S Finlayson K Tunçalp Ö Gülmezoglu AM Factors that influence the uptake of routine antenatal services by pregnant women: a qualitative evidence synthesis Cochrane Database Syst Rev 2016 10 CD012392 PMC6564082 31194903
  41. [48]Fretheim A Munabi-Babigumira S Oxman AD Lavis JN Lewin S SUPPORT tools for evidence-informed policymaking in health 6: Using research evidence to address how an option will be implemented Health Res Policy Syst 2009 7 Suppl 1 6 10.1186/1478-4505-7-S1-S6 20018113 PMC3271833 · DOI
  42. [50]Memon ZA Khan GN Soofi SB Baig IY Bhutta ZA Impact of a community-based perinatal and newborn preventive care package on perinatal and neonatal mortality in a remote mountainous district in Northern Pakistan BMC Pregnancy Childbirth 2015 15 106 10.1186/s12884-015-0538-8 25925407 PMC4446857 · DOI
  43. [52]Oxman AD Bjørndal A Becerra-Posada F Gibson M Block MA Haines A A framework for mandatory impact evaluation to ensure well informed public policy decisions Lancet 2010 375 427 431 10.1016/S0140-6736(09)61251-4 20113827 · DOI
  44. [53]Petkovic J Welch V Jacob MH Yoganathan M Ayala AP Cunningham H The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: a systematic review Implement Sci 2016 11 162 10.1186/s13012-016-0530-3 27938409 PMC5148903 · DOI

Cited by

Sorted by citations

Authors
for the GRADE Working Group, Moberg, J., Oxman, A. D., Rosenbaum, S., Schünemann, H. J., Guyatt, G., Flottorp, S., Glenton, C., Lewin, S., Morelli, A., Rada, G., & Alonso-Coello, P.
Affiliations
  • Norwegian Institute of Public Health (NO)
  • McMaster University (CA)
  • South African Medical Research Council (ZA)
Corresponding author
Moberg, Jenny
ORCID
Venue
Health Research Policy and Systems
Publisher
Springer Science and Business Media LLC
Published
2018-05-29
Volume
16
Issue
1
Pages
45-45
ISSN
1478-4505 (ISSN-L)
DOI
10.1186/s12961-018-0320-2
PMC
PMC5975536
OpenAlex
W2805824819
Cited by
376
References
3
Citations by year
2026: 41 · 2025: 46 · 2024: 66 · 2023: 52 · 2022: 52 · 2021: 45 · 2020: 54 · 2019: 15 · 2018: 5
FWCI
28.37
Type
journal article
Language
en
Open access
Licence
Locations
  • Health Research Policy and Systems (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Dipòsit Digital de Documents de la UAB (Universitat Autònoma de Barcelona) (submittedVersion, cc-by)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • NORA - Norwegian Open Research Archives (submittedVersion, other-oa)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
Funders
  • European Commission FP7 Program (258583)
  • South African Medical Research Council
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
health system decision-making, public health decision-making, evidence-informed policy, health policy recommendations, equity considerations, feasibility and acceptability, implementation considerations, monitoring and evaluation, guideline development, stakeholder engagement, DECIDE project
MeSH terms
Administrative Personnel, Decision Making, Delivery of Health Care, Health Policy, Humans, Policy Making, Public Health, Evidence-Based Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review