Scalable Decision Rules for Environmental Impact Studies: Effect Size, Type I, and Type II Errors

Abstract

Assessments of environmental impacts are being subject to greater scientific and legal scrutiny than ever before. The application of traditional statistical decision‐making criteria to questions of environmental impacts has become increasingly inadequate as society demands greater environmental accountability from economic development. In particular, impact assessment has inherited a preoccupation with Type I error rates that has pervaded ecological research, even though Type II errors are often equally severe in impact assessment. Estimation of Type II error rates and specification of critical effect sizes–or the magnitudes of impacts considered important–are mutually dependent. Consideration of Type II errors, therefore, requires the exact specification of an hypothesized impact, which is often difficult. Insistence on low rates of Type I error (e.g., α = 0.05) typically means that equivalent rates of Type II error can be realized only when effect sizes (ES) are very large or when very many samples are taken. Rather than adhering to a fixed, arbitrary, critical, Type I error rate, I propose a procedure by which the critical ES is given primacy. Statistical decision criteria are then selected according to the relative weighting of the perceived consequences of Type I or Type II errors. The critical Type I error rate is set by iteration to some multiple (k) of the estimated potential for Type II error, and the null hypothesis is rejected if that (variable) Type I probability is not exceeded. The value of k would be determined by the ratio of the consequences (e.g., costs) of Type II and Type I errors. The procedure focuses attention on the magnitudes of impacts considered important, and provides for statistical decisions based on the a priori consideration of the development and environmental costs of Type I and Type II errors. It also provides incentive for development proponents to support rigorous environmental monitoring.