On the use of systematic reviews to inform environmental policies
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Environmental research varies in its methodological quality, degree of bias, and relevance to policy questions. Using this heterogeneous, and sometimes polarised, research to inform environmental policies can be challenging. Policy-making in the healthcare field sometimes uses systematic reviews (SRs) to tackle these issues and present a comprehensive, policy-neutral, transparent and reproducible synthesis of the evidence. However, there is less familiarity with SRs in the environmental field. The aim of this article is to: (1) summarise the process of conducting SRs, using best practice methods from the healthcare field as an example, (2) explain the rationale behind each stage of conducting a SR, and (3) examine the prospects and challenges of using SRs to inform environmental policy. We conclude that existing SR protocols from healthcare can be, and have been, applied successfully to environmental research but some adaptations could improve the process. The literature search stage could be expedited by standardising the reporting and indexing of environmental studies, equivalent to that in the healthcare field. The consistency of the study appraisal stage of SRs could be augmented by refining the existing quality assessment tools used in the healthcare field, enhancing their ability to discriminate quality and risk of bias in non-randomised studies. Ultimately, the strength of evidence within SRs on environmental topics could be improved through more widespread use of randomised controlled trials as a research method, owing to their inherently lower risk of bias when conducted according to best practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Grading quality of evidence and strength of recommendations
(2004) · BMJ · doi:10.1136/bmj.328.7454.1490 · Cited by 8,477
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560
Assessing Risk of Bias in Included Studies
Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch8 · Cited by 3,396
What is ‘quality of evidence’ and why is it important to clinicians?
Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39490.551019.be · Cited by 3,250
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931 · cited 1×
“As this example suggests, the detailed specification of the review question requires consideration of several key components (Richardson et al., 1995; Counsell, 1997), including the types of populations (or participants), types of interventions and comparisons, and the types of outcomes that are of…”
BackgroundEvaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Analysing data and undertaking meta‐analyses
Deeks, J. J., Higgins, J. P., Altman, D. G., & on behalf of the Cochrane Statistical Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch10 · Cited by 2,672
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Thomas, B. H., Ciliska, D., Dobbins, M., & Micucci, S. (2004) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1524-475x.2004.04006.x · Cited by 2,118
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · cited 2×
“The Collaboration argue that while scales offer appealing simplicity; theoretical (Greenland and O’Rourke, 2001), and empirical evidence (Juni et al., 1999), suggests that their associations with intervention effect estimates are inconsistent and unpredictable (Balk et al., 2002; Emerson et al.,…”
BackgroundMethodAnalysing Data and Undertaking Meta‐Analyses
Deeks, J. J., Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch9 · Cited by 1,800
Impact of covert duplicate publication on meta-analysis: a case study
Tramèr, M. R., Reynolds, D. J. M., Moore, R. A., & McQuay, H. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.635 · Cited by 702
Bias in clinical intervention research.
Gluud, L. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj069 · Cited by 636
Schünemann, H. J., Jaeschke, R., Cook, D. J., Bria, W. F., El-Solh, A. A., Ernst, A., Fahy, B. F., Gould, M. K., Horan, K. L., Krishnan, J. A., et al. (2006) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200602-197st · Cited by 577
Systematic Reviews from Astronomy to Zoology: Myths and Misconceptions
Petticrew, M. (2001) · BMJ · doi:10.1136/bmj.322.7278.98 · Cited by 550
Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews
Counsell, C. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-5-199709010-00008 · Cited by 505 · cited 1×
“…this example suggests, the detailed specification of the review question requires consideration of several key components (Richardson et al., 1995; Counsell, 1997), including the types of populations (or participants), types of interventions and comparisons, and the types of outcomes that are of…”
BackgroundBalk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478 · cited 1×
“…while scales offer appealing simplicity; theoretical (Greenland and O’Rourke, 2001), and empirical evidence (Juni et al., 1999), suggests that their associations with intervention effect estimates are inconsistent and unpredictable (Balk et al., 2002; Emerson et al., 1990; Schulz et al., 1995).”
BackgroundThe Science of Reviewing Researcha
OXMAN, A. D., & GUYATT, G. H. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26342.x · Cited by 472
Selecting Studies and Collecting Data
Higgins, J. P., & Deeks, J. J. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch7 · Cited by 467
Greenland, S. (2001) · Biostatistics · doi:10.1093/biostatistics/2.4.463 · Cited by 359 · cited 1×
“The Collaboration argue that while scales offer appealing simplicity; theoretical (Greenland and O’Rourke, 2001), and empirical evidence (Juni et al., 1999), suggests that their associations with intervention effect estimates are inconsistent and unpredictable (Balk et al., 2002; Emerson et al.,…”
BackgroundRisk of bias versus quality assessment of randomised controlled trials: cross sectional study
Hartling, L., Ospina, M., Liang, Y., Dryden, D. M., Hooton, N., Krebs Seida, J., & Klassen, T. P. (2009) · BMJ · doi:10.1136/bmj.b4012 · Cited by 348
Edwards, P., Clarke, M., DiGuiseppi, C., Pratap, S., Roberts, I., & Wentz, R. (2002) · Statistics in Medicine · doi:10.1002/sim.1190 · Cited by 340 · cited 1×
“Using at least two authors, searching independently, reduces the possibility that relevant reports will be discarded (Edwards et al., 2002).”
BackgroundGetting the methods right--the foundation of patient-centered outcomes research
Gabriel, S. E., & Normand, S.-L. T. (2012) · New England Journal of Medicine · doi:10.1056/nejmp1207437 · Cited by 252
Hartling, L., Hamm, M. P., Milne, A., Vandermeer, B., Santaguida, P. L., Ansari, M., Tsertsvadze, A., Hempel, S., Shekelle, P., & Dryden, D. M. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.07.005 · Cited by 173
Identifying observational studies of surgical interventions in MEDLINE and EMBASE
Fraser, C., Murray, A., & Burr, J. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-41 · Cited by 58
Limited search strategies were effective in finding relevant nonrandomized studies
Furlan, A. D., Irvin, E., & Bombardier, C. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.004 · Cited by 54 · cited 1×
“Some review authors have tried to develop, optimise and ‘validate’ search strategies for NRS (Wieland and Dickersin, 2005; Fraser et al., 2006; Furlan et al., 2006; Golder et al., 2006a,b), and there is a dedicated NRS Methods Group who provide guidance to support authors who are considering…”
Background
Cited by
Do European agroforestry systems enhance biodiversity and ecosystem services? A meta-analysis
Torralba, M., Fagerholm, N., Burgess, P. J., Moreno, G., & Plieninger, T. (2016) · Agriculture, Ecosystems & Environment · doi:10.1016/j.agee.2016.06.002 · Cited by 689
Eight problems with literature reviews and how to fix them
Haddaway, N. R., Bethel, A., Dicks, L. V., Koricheva, J., Macura, B., Petrokofsky, G., Pullin, A. S., Savilaakso, S., & Stewart, G. B. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-01295-x · Cited by 256
Cyber security in the maritime industry: a systematic survey of recent advances and future trends
Ben Farah, M. A., Ukwandu, E., Hindy, H., Brosset, D., Bures, M., Andonovic, I., & Bellekens, X. (2022) · Information · doi:10.3390/info13010022 · Cited by 175
The reliability of evidence review methodology in environmental science and conservation
O’Leary, B. C., Kvist, K., Bayliss, H. R., Derroire, G., Healey, J. R., Hughes, K., Kleinschroth, F., Sciberras, M., Woodcock, P., & Pullin, A. S. (2016) · Environmental Science & Policy · doi:10.1016/j.envsci.2016.06.012 · Cited by 65
Using machine learning to disentangle homonyms in large text corpora
Roll, U., Correia, R. A., & Berger‐Tal, O. (2018) · Conservation Biology · doi:10.1111/cobi.13044 · Cited by 46
Metadata
- Authors
- Bilotta, G. S., Milner, A. M., & Boyd, I.
- Venue
- Environmental Science & Policy
- Publisher
- Elsevier BV
- Published
- 2014-10
- Volume
- 42
- Pages
- 67-77
- ISSN
- 1462-9011 (ISSN-L)
- OpenAlex
- W2161901522
- Cited by
- 137
- References
- 27
- FWCI
- 4.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.