The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
GRADE: Assessing the quality of evidence in environmental and occupational health
Open accessAbstract
There is high demand in environmental health for adoption of a structured process that evaluates and integrates evidence while making decisions and recommendations transparent. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework holds promise to address this demand. For over a decade, GRADE has been applied successfully to areas of clinical medicine, public health, and health policy, but experience with GRADE in environmental and occupational health is just beginning. Environmental and occupational health questions focus on understanding whether an exposure is a potential health hazard or risk, assessing the exposure to understand the extent and magnitude of risk, and exploring interventions to mitigate exposure or risk. Although GRADE offers many advantages, including its flexibility and methodological rigor, there are features of the different sources of evidence used in environmental and occupational health that will require further consideration to assess the need for method refinement. An issue that requires particular attention is the evaluation and integration of evidence from human, animal, in vitro, and in silico (computer modelling) studies when determining whether an environmental factor represents a potential health hazard or risk. Assessment of the hazard of exposures can produce analyses for use in the GRADE evidence-to-decision (EtD) framework to inform risk-management decisions about removing harmful exposures or mitigating risks. The EtD framework allows for grading the strength of the recommendations based on judgments of the certainty in the evidence (also known as quality of the evidence), as well as other factors that inform recommendations such as social values and preferences, resource implications, and benefits. GRADE represents an untapped opportunity for environmental and occupational health to make evidence-based recommendations in a systematic and transparent manner. The objectives of this article are to provide an overview of GRADE, discuss GRADE’s applicability to environmental health, and identify priority areas for method assessment and development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Self-citationGRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Self-citationGRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citationSYRCLE’s risk of bias tool for animal studies
Hooijmans, C. R., Rovers, M. M., de Vries, R. B., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-43 · Cited by 4,415
Self-citationGRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology
Guyatt, G. H., Oxman, A. D., Schünemann, H. J., Tugwell, P., & Knottnerus, A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.011 · Cited by 3,094
Self-citationThe well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931
GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citationGRADE guidelines: 2. Framing the question and deciding on important outcomes
Guyatt, G. H., Oxman, A. D., Kunz, R., Atkins, D., Brozek, J., Vist, G., Alderson, P., Glasziou, P., Falck-Ytter, Y., & Schünemann, H. J. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.012 · Cited by 1,911
Self-citationGRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citationGrading quality of evidence and strength of recommendations for diagnostic tests and strategies
Schünemann, H. J., Oxman, A. D., Brozek, J., Glasziou, P., Jaeschke, R., Vist, G. E., Williams, J. W., Kunz, R., Craig, J., Montori, V. M., et al. (2008) · BMJ · doi:10.1136/bmj.39500.677199.ae · Cited by 1,782
Self-citationToxicity Testing in the 21st Century: A Vision and a Strategy
Krewski, D., Acosta, D., Andersen, M., Anderson, H., Bailar, J. C., Boekelheide, K., Brent, R., Charnley, G., Cheung, V. G., Green, S., et al. (2010) · Journal of Toxicology and Environmental Health, Part B · doi:10.1080/10937404.2010.483176 · Cited by 1,688
Andrews, J., Guyatt, G., Oxman, A. D., Alderson, P., Dahm, P., Falck-Ytter, Y., Nasser, M., Meerpohl, J., Post, P. N., Kunz, R., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.03.013 · Cited by 1,416
Self-citationToxicity testing in the 21st century: A vision and a strategy
Gibb, S. (2008) · Reproductive Toxicology · doi:10.1016/j.reprotox.2007.10.013 · Cited by 1,257
Atkins, D., Eccles, M., Flottorp, S., Guyatt, G. H., Henry, D., Hill, S., Liberati, A., O'Connell, D., Oxman, A. D., Phillips, B., et al. (2004) · BMC Health Services Research · doi:10.1186/1472-6963-4-38 · Cited by 1,235 · cited 1×
“GRADE is widely used internationally to address topics related to clinical medicine, public health, and health policy (Atkins et al., 2004; Guyatt et al., 2011d, 2008; Schünemann et al., 2008), including by programswithin theU.”
MethodSelf-citationIorio, A., Spencer, F. A., Falavigna, M., Alba, C., Lang, E., Burnand, B., McGinn, T., Hayden, J., Williams, K., Shea, B., et al. (2015) · BMJ · doi:10.1136/bmj.h870 · Cited by 880
Self-citationChapter 12: Interpreting results and drawing conclusions
Schünemann, H. J., Oxman, A. D., Jonathan, J., & Glasziou, P. (2011) · Cited by 870
Self-citationShift work and vascular events: systematic review and meta-analysis
Vyas, M. V., Garg, A. X., Iansavichus, A. V., Costella, J., Donner, A., Laugsand, L. E., Janszky, I., Mrkobrada, M., Parraga, G., & Hackam, D. G. (2012) · BMJ · doi:10.1136/bmj.e4800 · Cited by 840
Higgins, J. (2011) · Medical Entomology and Zoology · Cited by 816
Authority, E. G. F. T. C. O. S. R. E. F. S. (2010) · EFSA Journal · doi:10.2903/j.efsa.2010.1637 · Cited by 693
Improving the Human Hazard Characterization of Chemicals: A Tox21 Update
Tice, R. R., Austin, C. P., Kavlock, R. J., & Bucher, J. R. (2013) · Environmental Health Perspectives · doi:10.1289/ehp.1205784 · Cited by 618
Interpreting results and drawing conclusions
Schünemann, H. J., Vist, G. E., Higgins, J. P., Santesso, N., Deeks, J. J., Glasziou, P., Akl, E. A., Guyatt, G. H., & on behalf of the Cochrane GRADEing Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch15 · Cited by 595
Self-citationThe Toxicity Data Landscape for Environmental Chemicals
Judson, R., Richard, A., Dix, D. J., Houck, K., Martin, M., Kavlock, R., Dellarco, V., Henry, T., Holderman, T., Sayre, P., et al. (2009) · Environmental Health Perspectives · doi:10.1289/ehp.0800168 · Cited by 508
Kavlock, R., Chandler, K., Houck, K., Hunter, S., Judson, R., Kleinstreuer, N., Knudsen, T., Martin, M., Padilla, S., Reif, D., et al. (2012) · Chemical Research in Toxicology · doi:10.1021/tx3000939 · Cited by 494
Johnson, P. I., Sutton, P., Atchley, D. S., Koustas, E., Lam, J., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307893 · Cited by 449 · cited 4×
“Work to-date from the Navigation Guide, NTP, and WHO show the GRADE framework is sufficiently flexible to support use now (Johnson et al., 2013, 2014; Koustas et al., 2014; Lam et al., 2014; NTP, 2015; WHO, 2014a); however, areas for further method assessment have been identified.”
BackgroundMethodInfluentialSelf-citationAssessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions
Viswanathan, M., Patnode, C. D., Berkman, N. D., Bass, E. B., Chang, S., Hartling, L., Murad, M. H., Treadwell, J. R., & Kane, R. L. (2017) · doi:10.23970/ahrqepcmethguide2 · Cited by 448
Rooney, A. A., Boyles, A. L., Wolfe, M. S., Bucher, J. R., & Thayer, K. A. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307972 · Cited by 434
Self-citationWoodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427
Self-citationKoustas, E., Lam, J., Sutton, P., Johnson, P. I., Atchley, D. S., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307177 · Cited by 349
Self-citationLetters, numbers, symbols and words: how to communicate grades of evidence and recommendations.
Schünemann, H. J., Best, D., Vist, G. E., & Oxman, A. D. (2003) · PubMed Central · Cited by 289
Self-citationThe GRADE approach and Bradford Hill's criteria for causation
Schünemann, H., Hill, S., Guyatt, G., Akl, E. A., & Ahmed, F. (2011) · Journal of Epidemiology and Community Health · doi:10.1136/jech.2010.119933 · Cited by 262
Self-citationNight-shift work and breast cancer – a systematic review and meta-analysis
Ijaz, S. I., Verbeek, J. H., Seidler, A., Lindbohm, M.-L., Ojajärvi, A., Orsini, N., Costa, G., & Neuvonen, K. (2013) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.3371 · Cited by 242
Self-citationBruce, N., Pope, D., Rehfuess, E., Balakrishnan, K., Adair-Rohani, H., & Dora, C. (2015) · Atmospheric Environment · doi:10.1016/j.atmosenv.2014.08.064 · Cited by 238
Self-citationLam, J., Koustas, E., Sutton, P., Johnson, P. I., Atchley, D. S., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307923 · Cited by 214
Self-citationInterventions to prevent occupational noise-induced hearing loss
Tikka, C., Verbeek, J. H., Kateman, E., Morata, T. C., Dreschler, W. A., & Ferrite, S. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006396.pub4 · Cited by 214
Self-citationDECIDE Consortium, Treweek, S., Oxman, A. D., Alderson, P., Bossuyt, P. M., Brandt, L., Brożek, J., Davoli, M., Flottorp, S., Harbour, R., et al. (2013) · Implementation Science · doi:10.1186/1748-5908-8-6 · Cited by 201
Self-citationIARC Monographs: 40 years of evaluating carcinogenic hazards to humans
Pearce, N., Blair, A., Vineis, P., Ahrens, W., Andersen, A., Anto, J. M., Armstrong, B. K., Baccarelli, A. A., Beland, F. A., Berrington, A., et al. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1409149 · Cited by 146
Woodruff, T. J., Sutton, P., & The Navigation Guide Work Group (2011) · Health Affairs · doi:10.1377/hlthaff.2010.1219 · Cited by 134
Self-citationQuality assessment tools for evidence from environmental science
Bilotta, G. S., Milner, A. M., & Boyd, I. L. (2014) · Environmental Evidence · doi:10.1186/2047-2382-3-14 · Cited by 113
IARC: diesel engine exhaust carcinogenic.
(2012) · PubMed · Cited by 108
Rehfuess, E. A., & Akl, E. A. (2013) · BMC Public Health · doi:10.1186/1471-2458-13-9 · Cited by 97
Self-citationAhmed, F., Temte, J. L., Campos-Outcalt, D. E., & Schünemann, H. J. (2011) · Vaccine · doi:10.1016/j.vaccine.2011.08.005 · Cited by 93
Self-citationWhaley, P., Halsall, C. J., Ågerstrand, M., Aiassa, E., Benford, D., Bilotta, G. S., Coggon, D., Collins, C. D., Dempsey, C., Duarte‐Davidson, R., et al. (2015) · Environment International · doi:10.1016/j.envint.2015.11.002 · Cited by 92
Self-citationIntroducing GRADE across the NICE clinical guideline program
Thornton, J., Alderson, P., Tan, T., Turner, C., Latchem, S., Shaw, E. J., Ruiz, F. J., Reken, S., Mugglestone, M. A., Hill, J., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.12.007 · Cited by 69
Interventions to prevent occupational noise-induced hearing loss
Verbeek, J. H. A. M., Kateman, E., Morata, T. C., Dreschler, W. A., & Tikka, C. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006396.pub3 · Cited by 69
Self-citationUncertainties in baseline risk estimates and confidence in treatment effects
Spencer, F. A., Iorio, A., You, J. J., Murad, M. H., Schünemann, H. J., Vandvik, P. O., Crowther, M. A., Pottie, K. C., Lang, E., Meerpohl, J. J., et al. (2012) · BMJ · doi:10.1136/bmj.e7401 · Cited by 56
Self-citationEvidence from Toxicology: The Most Essential Science for Prevention
Mandrioli, D., & Silbergeld, E. K. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1509880 · Cited by 53
Self-citationQuality of evidence is a key determinant for making a strong GRADE guidelines recommendation
Djulbegović, B., Kumar, A., Kaufman, R. M., Tobian, A. A. R., & Guyatt, G. H. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.12.015 · Cited by 50
Self-citationAiassa, E., Higgins, J. P. T., Frampton, G. K., Greiner, M., Afonso, A., Amzal, B., Deeks, J. J., Dorne, J. L. C. M., Glanville, J., Löveï, G. L., et al. (2014) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2013.769933 · Cited by 47
Fetal growth and maternal glomerular filtration rate: a systematic review
Vesterinen, H. M., Johnson, P. I., Atchley, D. S., Sutton, P., Lam, J., Zlatnik, M. G., Sen, Ś., & Woodruff, T. J. (2014) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.3109/14767058.2014.980809 · Cited by 42
Self-citationSystematic review of the human health effects of wind farms
Merlin, T. W. J., Newton, S. A., Ellery, B., Milverton, J., & Farah, C. (2013) · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · Cited by 31
Evidence-based toxicology: Strait is the gate, but the road is worth taking
Silbergeld, E. K. (2013) · ALTEX · doi:10.14573/altex.2013.1.067 · Cited by 20
Self-citationSchünemann, H. J. (2013) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2013.05.002 · Cited by 12
Self-citationImplementing systematic review in toxicological profiles: ATSDR and NIEHS/NTP collaboration.
Murray, H., & Thayer, K. A. (2014) · PubMed · Cited by 9
Self-citationBruce, N., Dora, C. F. C., Krzyżanowski, M., Adair‐Rohani, H., Morawska, L., & Wangchuk, T. (2013) · QUT ePrints (Queensland University of Technology) · Cited by 8
Self-citation260 Interventions to prevent occupational noise-induced hearing loss - a Cochrane Review
Verbeek, V. J., Kateman, Morata, Dreschler, & Mischke (2013) · Occupational and Environmental Medicine · doi:10.1136/oemed-2013-101717.260 · Cited by 6
Cited by
Morgan, R. L., Whaley, P., Thayer, K. A., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.07.015 · Cited by 1,282
Self-citationHaddaway, N. R., Macura, B., Whaley, P., & Pullin, A. S. (2018) · Environmental Evidence · doi:10.1186/s13750-018-0121-7 · Cited by 946
Overview of known plastic packaging-associated chemicals and their hazards
Groh, K. J., Backhaus, T., Carney-Almroth, B., Geueke, B., Inostroza, P. A., Lennquist, A., Leslie, H. A., Maffini, M., Slunge, D., Trasande, L., et al. (2019) · Science of The Total Environment · doi:10.1016/j.scitotenv.2018.10.015 · Cited by 913
Schünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911
Self-citationVan Kempen, E., Casas, M., Pershagen, G., & Foraster, M. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15020379 · Cited by 619
Metadata
- Authors
- Morgan, R. L., Thayer, K. A., Bero, L., Bruce, N., Falck-Ytter, Y., Ghersi, D., Guyatt, G., Hooijmans, C., Langendam, M., Mandrioli, D., Mustafa, R. A., Rehfuess, E. A., Rooney, A. A., Shea, B., Silbergeld, E. K., Sutton, P., Wolfe, M. S., Woodruff, T. J., Verbeek, J. H., Holloway, A. C., Santesso, N., & Schünemann, H. J.
- Venue
- Environment International
- Publisher
- Elsevier BV
- Published
- 2016-07
- Volume
- 92-93
- Pages
- 611-616
- ISSN
- 0160-4120 (ISSN-L)
- OpenAlex
- W2287802286
- Cited by
- 277
- References
- 57
- FWCI
- 21.16
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.