The DOI matches its Crossref record; 61 of the 63 references it cites were found in a registry or an index.
Analyzing cause and effect in environmental assessments: using weighted evidence from the literature
Abstract
Sound decision making in environmental research and management requires an understanding of causal relationships between stressors and ecological responses. However, demonstrating cause–effect relationships in natural systems is challenging because of difficulties with natural variability, performing experiments, lack of replication, and the presence of confounding influences. Thus, even the best-designed study may not establish causality. We describe a method that uses evidence available in the extensive published ecological literature to assess support for cause–effect hypotheses in environmental investigations. Our method, called Eco Evidence, is a form of causal criteria analysis—a technique developed by epidemiologists in the 1960s—who faced similar difficulties in attributing causation. The Eco Evidence method is an 8-step process in which the user conducts a systematic review of the evidence for one or more cause–effect hypotheses to assess the level of support for an overall question. In contrast to causal criteria analyses in epidemiology, users of Eco Evidence use a subset of criteria most relevant to environmental investigations and weight each piece of evidence according to its study design. Stronger studies contribute more to the assessment of causality, but weaker evidence is not discarded. This feature is important because environmental evidence is often scarce. The outputs of the analysis are a guide to the strength of evidence for or against the cause–effect…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To evaluate the results, we applied rules adapted from Norris et al. (2012) and Greet et al. (2011), and adapted the four-element conclusion framework developed by the U.S. Department of Health and Human Services as follows:”
From Evidence‐based evaluation of the cumulative effects of ecosystem restoration (2016) · cites this paper as Norris et al. (2012)
In the citing paper
“Studies associated with different disciplines had selected various subsets of Hill's criteria, (e.g., Norris et al. 2012).”
From Evidence‐based evaluation of the cumulative effects of ecosystem restoration (2016) · cites this paper as Norris et al. 2012
In the citing paper
“Like others have reported (Greet et al. 2011, Norris et al. 2012, Webb et al. 2012), evidence-based assessment of literature allowed a much larger proportion of the body of evidence to be considered than would a quantitative meta-analysis.”
From Evidence‐based evaluation of the cumulative effects of ecosystem restoration (2016) · cites this paper as Norris et al. 2012
Citations
Cites
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,804
The Logic of Scientific Discovery
Bhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732-9 · Cited by 7,765
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974
Experiments in ecology their logical design and interpretation using analysis of variance
Underwood, A. J. (1997) · Cited by 3,799
Platt, J. R. (1964) · Science · doi:10.1126/science.146.3642.347 · Cited by 2,907
The need for evidence-based conservation
Sutherland, W. J., Pullin, A. S., Dolman, P. M., & Knight, T. M. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2004.03.018 · Cited by 1,822
Sampling Design and Statistical Methods for Environmental Biologists
Green, R. H. (1979) · Andalas University Repository (Andalas University) · Cited by 1,557
The Logic of Scientific Discovery
Popper, K. (2005) · Routledge eBooks · doi:10.4324/9780203994627 · Cited by 1,195
Guidelines for systematic review in Conservation and environmental management
PULLIN, A. S., & STEWART, G. B. (2006) · Conservation Biology · doi:10.1111/j.1523-1739.2006.00485.x · Cited by 1,137
Welfare., E. (1964) · Cited by 1,115
Realism and the Aim of Science
Popper, K. R. (2013) · Routledge eBooks · doi:10.4324/9780203713969 · Cited by 1,073
Best evidence synthesis: An intelligent alternative to meta-analysis
Slavin, R. E. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00097-a · Cited by 951
Underwood, A. J. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511806407 · Cited by 862
Meta-analysis: Combining the Results of Independent Experiments
Gurevitch, J., & Hedges, L. V. (2001) · Oxford University Press eBooks · doi:10.1093/oso/9780195131871.003.0018 · Cited by 731
Wellmann, K. F. (1964) · PubMed · Cited by 691
Causation and disease: the Henle-Koch postulates revisited.
Evans, A. S. (1976) · PubMed Central · Cited by 635
Schottenfeld, D. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf028 · Cited by 627
Recent developments in meta‐analysis
Sutton, A. J., & Higgins, J. P. T. (2007) · Statistics in Medicine · doi:10.1002/sim.2934 · Cited by 601
Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy
Song, F., Khan, K. S., Dinnes, J., & Sutton, A. J. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.88 · Cited by 485
Effectiveness in Conservation Practice: Pointers from Medicine and Public Health
Pullin, A. S., & Knight, T. (2001) · Conservation Biology · doi:10.1111/j.1523-1739.2001.99499.x · Cited by 470
Causal inference in disease ecology: investigating ecological drivers of disease emergence
Plowright, R. K., Sokolow, S. H., Gorman, M. E., Daszak, P., & Foley, J. E. (2008) · Frontiers in Ecology and the Environment · doi:10.1890/070086 · Cited by 350
Assessing cause and effect of multiple stressors on marine systems
Adams, S. M. (2004) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2004.11.040 · Cited by 255
The Importance of Replication in Wildlife Research
Johnson, D. H. (2002) · Journal of Wildlife Management · doi:10.2307/3802926 · Cited by 254
Practical causal inference for ecoepidemiologists
Fox, G. A. (1991) · Journal of Toxicology and Environmental Health · doi:10.1080/15287399109531535 · Cited by 209
IDENTIFYING ECOLOGICAL CHANGE AND ITS CAUSES: A CASE STUDY ON CORAL REEFS
Fabricius, K., & De’ath, G. (2004) · Ecological Applications · doi:10.1890/03-5320 · Cited by 168
GREET, J., ANGUS WEBB, J., & COUSENS, R. D. (2011) · Freshwater Biology · doi:10.1111/j.1365-2427.2011.02564.x · Cited by 121
Self-citationThe role of causal criteria in causal inferences: Bradford Hill's "aspects of association"
Ward, A. C. (2009) · Epidemiologic Perspectives & Innovations · doi:10.1186/1742-5573-6-2 · Cited by 120
Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related
Weed, D. L. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.3.387 · Cited by 120
A METHODOLOGY FOR INFERRING THE CAUSES OF OBSERVED IMPAIRMENTS IN AQUATIC ECOSYSTEMS
Suter, G. W., Norton, S. B., & Cormier, S. M. (2002) · Environmental Toxicology and Chemistry · doi:10.1002/etc.5620210602 · Cited by 119
Popper's Philosophy for Epidemiologists
Buck, C. W. (1975) · International Journal of Epidemiology · doi:10.1093/ije/4.3.159 · Cited by 115
Pullin, A. S., Knight, T. M., & Watkinson, A. R. (2009) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2009.01704.x · Cited by 104
THE LOGIC OF SIR KARL POPPER AND THE PRACTICE OF EPIDEMIOLOGY
Susser, M. W. (1986) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114446 · Cited by 100
The practice of causal inference in cancer epidemiology.
Weed, D. L., & GORELIC, L. S. (1996) · PubMed · Cited by 98
Why and how to combine evidence in environmental assessments: Weighing evidence and building cases
Suter, G. W., & Cormier, S. M. (2011) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2010.12.029 · Cited by 97
Weed, D. (1997) · International Journal of Epidemiology · doi:10.1093/ije/26.6.1137 · Cited by 97
Causal Inference in Environmental Impact Studies
Beyers, D. W. (1998) · Journal of the North American Benthological Society · doi:10.2307/1468339 · Cited by 83
Lowell, R. B., Culp, J. M., & Dubé, M. G. (2000) · Environmental Toxicology and Chemistry · doi:10.1002/etc.5620190452 · Cited by 80
On the Origin of Hillʼs Causal Criteria
Morabia, A. (1991) · Epidemiology · doi:10.1097/00001648-199109000-00010 · Cited by 75
Non‐significant results in ecology: a burden or a blessing in disguise?
(2003) · Oikos · doi:10.1034/j.1600-0579.2003.12353.x · Cited by 75
Meta‐analysis in Ecology: Concepts, Statistics, and Applications
Osenberg, C., Sarnelle, O., & Goldberg, D. (1999) · Ecology · doi:10.1890/0012-9658(1999)080[1103:maiecs]2.0.co;2 · Cited by 70
Rules of inference in epidemiology
Susser, M. W. (1986) · Regulatory Toxicology and Pharmacology · doi:10.1016/0273-2300(86)90029-2 · Cited by 69
Burkhardt-Holm, P., & Scheurer, K. (2007) · Aquatic Sciences · doi:10.1007/s00027-006-0841-6 · Cited by 62
Assessing the attributable risks, relative risks, and regional extents of aquatic stressors
Sickle, J. V. V., & Paulsen, S. G. (2008) · Journal of the North American Benthological Society · doi:10.1899/07-152.1 · Cited by 60
The Science and Philosophy of a Method for Assessing Environmental Causes
Suter, G. W., Norton, S. B., & Cormier, S. M. (2010) · Human and Ecological Risk Assessment: An International Journal · doi:10.1080/10807030903459254 · Cited by 59
CADDIS: The Causal Analysis/Diagnosis Decision Information System
Norton, S. B., Cormier, S. M., Suter, G. W., Schofield, K. A., Yuan, L. L., Shaw‐Allen, P., & Ziegler, C. R. (2008) · Springer eBooks · doi:10.1007/978-0-387-09722-0_17 · Cited by 54
Causal Characteristics for Ecoepidemiology
Cormier, S. M., Suter, G. W., & Norton, S. B. (2010) · Human and Ecological Risk Assessment An International Journal · doi:10.1080/10807030903459320 · Cited by 52
Non-significant results in ecology: a burden or a blessing in disguise?
Koricheva, J. (2003) · Cited by 46
Using Relative Risk to Compare the Effects of Aquatic Stressors at a Regional Scale
Sickle, J. V. V., Stoddard, J. L., Paulsen, S. G., & Olsen, A. R. (2006) · Environmental Management · doi:10.1007/s00267-005-0240-0 · Cited by 45
Weed, D. L. (2002) · Toxicology · doi:10.1016/s0300-483x(02)00476-6 · Cited by 36
Ecological Responses to Flow Alteration: Assessing Causal Relationships with Eco Evidence
Webb, J. A., Nichols, S. J., Norris, R. H., Stewardson, M. J., Wealands, S. R., & Lea, P. (2011) · Wetlands · doi:10.1007/s13157-011-0249-5 · Cited by 34
Self-citationCormier, S., Norton, S., Suter, G., Altfater, D., & Counts, B. (2002) · Environmental Toxicology and Chemistry · doi:10.1002/etc.5620210604 · Cited by 25
Collier, T. K. (2003) · Human and Ecological Risk Assessment An International Journal · doi:10.1080/713609862 · Cited by 24
Nichols, J. R., Webb, J. A., Norris, R., & Stewardson, M. J. (2011) · Cited by 24
Self-citation(2011) · International Congress on Modelling and Simulation (MODSIM) · doi:10.36334/modsim.2011.e15.webb · Cited by 18
Multiple Lines and Levels of Evidence for Detecting Ecological Responses to Management Intervention
Norris, R., Liston, P., Mugodo, J., & Nichols, S. J. (2005) · University of Canberra Research Portal · Cited by 16
Self-citationCausal Assessment of Biological Impairment in the Little Floyd River, Iowa, USA
Haake, D. M., Wilton, T., Krier, K., Stewart, A. J., & Cormier, S. M. (2010) · Human and Ecological Risk Assessment An International Journal · doi:10.1080/10807030903459544 · Cited by 13
Rothman, K. J., Huybrechts, K. F., & Murray, E. J. (2024) · doi:10.1093/oso/9780197751541.001.0001 · Cited by 12
Assessment of Probable Causes of Reduced Aquatic Life in the Touchet River, Washington, USA
Wiseman, C. D., Lemoine, M., & Cormier, S. M. (2010) · Human and Ecological Risk Assessment An International Journal · doi:10.1080/10807030903459429 · Cited by 11
Sampling Design and Statistical Methods for Environmental Biologists.
White, R. J., & Green, R. H. (1981) · Journal of Ecology · doi:10.2307/2259836 · Cited by 10
Fine sediment in rivers : scale of ecological outcomes
Harrison, E. T. (2021) · University of Canberra Research Portal · doi:10.26191/x90w-ev66 · Cited by 4
Self-citationEvidence-based model structure: The role of causal analysis in hydro-ecological modelling
Wealands, S., Webb, J. A., & Stewardson, M. J. (2009) · Cited by 3
Self-citation
Research works printed but not linked: 9
- DOWNES B. J., Monitoring ecological impacts: concepts and practice in flowing waters, 10.1017/CBO9780511542015 · DOI · Paper page
- EVANS A. S., 1976, Yale Journal of Biology and Medicine, 49, 175
- EWATER CRC E)., Eco Evidence: a systematic approach to evaluate causality in environmental science. eWater Cooperative Research Centre
- GREENLAND S., Pages 643-673 in K
- ROTHMAN K. J., Modern epidemiology
- STEVENS A., The effectiveness revolution and public health. Pages 197-225 in G
- TUGWELL B., Assessing claims of causation. Pages 356-387 in R
- WEBB J. A., MODSIM2011 International Congress on Modelling and Simulation. Modelling and Simulation Society of Australia and New Zealand, Australia (in press b).
- WEED D. L., 1996, Cancer Epidemiology Biomarkers and Prevention, 5, 303
Cited by
Sorted by citations
Simple study designs in ecology produce inaccurate estimates of biodiversity responses
Christie, A. P., Amano, T., Martin, P. A., Shackelford, G. E., Simmons, B. I., & Sutherland, W. J. (2019) · Journal of Applied Ecology · doi:10.1111/1365-2664.13499 · Cited by 384
Hering, D., Carvalho, L., Argillier, C., Beklioğlu, M., Borja, Á., Cardoso, A. C., Duel, H., Ferreira, T., Globevnik, L., Hanganu, J., et al. (2014) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2014.06.106 · Cited by 304
Ecological Assessments with Algae: A Review and Synthesis
Stevenson, J. (2014) · Journal of Phycology · doi:10.1111/jpy.12189 · Cited by 273
The enemy release hypothesis as a hierarchy of hypotheses
Heger, T., & Jeschke, J. M. (2014) · Oikos · doi:10.1111/j.1600-0706.2013.01263.x · Cited by 184
Vilizzi, L., Tarkan, A. S., & Copp, G. H. (2015) · Reviews in Fisheries Science & Aquaculture · doi:10.1080/23308249.2015.1051214 · Cited by 153
Metadata
- Authors
- Norris, R. H., Webb, J. A., Nichols, S. J., Stewardson, M. J., & Harrison, E. T.
- Affiliations
- University of Canberra (AU)
- The University of Melbourne (AU)
- Corresponding author
- Norris, R. H.
- Webb, J. A.
- Nichols, S. J.
- Stewardson, M. J.
- Harrison, E. T.
- Venue
- Freshwater Science
- Publisher
- University of Chicago Press
- Published
- 2012-03
- Volume
- 31
- Issue
- 1
- Pages
- 5-21
- ISSN
- 2161-9549 (ISSN-L)
- 2161-9565
- OpenAlex
- W2127674064
- Cited by
- 113
- References
- 63
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 5 · 2023: 9 · 2022: 8 · 2021: 5 · 2020: 4 · 2019: 7 · 2018: 8 · 2017: 12 · 2016: 9 · 2015: 9 · 2014: 10 · 2013: 9 · 2012: 6
- FWCI
- 12.42
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Freshwater Science (publishedVersion)
- University of Canberra Research Portal (submittedVersion)
- BioOne Complete (BioOne) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Environmental Science › Management, Monitoring, Policy and Law › Environmental and Social Impact Assessments
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- causal inference, causal relationship, evidence synthesis, systematic reviews, quality of evidence, knowledge gaps, environmental management decision-making, evidence-based decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review