The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Assessing Effects of Unreplicated Perturbations: No Simple Solutions
Abstract
We address the task of determining the effects, on mean population density or other parameters, of an unreplicated perturbation, such as arises in environmental assessments and some ecosystem—level experiments. Our context is the Before—After—Control—Impact—Pairs design (BACIP): on several dates Before and After the perturbation, samples are collected simultaneously at both the Impact site and a nearby "Control." One approach is to test whether the mean of the Impact—Control difference has changed from Before to After the perturbation. If a conventional test is used, checks of its assumptions are an important and messy part of the analysis, since BACIP data do not necessarily satisfy them. It has been suggested that these checks are not needed for randomization tests, because they are insensitive to some of these assumptions and can be adjusted to allow for others. A major aim of this paper is to refute this suggestion: there is no panacea for the difficult and messy technical problems in the analysis of data from assessments or unreplicated experiments. We compare the randomization t test with the standard t test and the modified (Welch—Satterthwaite—Aspin) t test, which allows for unequal variances. We conclude that the randomization t test is less likely to yield valid inferences than is the Welch t test, because it requires identical distributions for small sample sizes and either equal variances or equal sample sizes for larger ones. The formal requirement of Normality is not crucial to the Welch t test. Both parametric and randomization tests require that time and location effects be additive and the Impact—Control differences on different dates be independent. These assumptions should be tested; if they are seriously wrong, alternative analyses are needed. This will often require a long time series of data. Finally, for assessing the importance of a perturbation, the P value of a hypothesis test is rarely as useful as an estimate of the size of the effect. Especially if effect size varies with time and conditions, flexible estimation methods with approximate answers are preferable to formally exact P values.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An introduction to probability theory and its applications
(1958) · Journal of the Franklin Institute · doi:10.1016/0016-0032(58)90688-4 · Cited by 29,269
An Analysis of Transformations
Box, G. E. P., & Cox, D. R. (1964) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1964.tb00553.x · Cited by 15,052
Pseudoreplication and the Design of Ecological Field Experiments
Hurlbert, S. H. (1984) · Ecological Monographs · doi:10.2307/1942661 · Cited by 8,377
On a measure of lack of fit in time series models
LJUNG, G. M., & BOX, G. E. P. (1978) · Biometrika · doi:10.1093/biomet/65.2.297 · Cited by 6,668
TESTING FOR SERIAL CORRELATION IN LEAST SQUARES REGRESSION. II
DURBIN, J., & WATSON, G. S. (1951) · Biometrika · doi:10.1093/biomet/38.1-2.159 · Cited by 3,619
Cascading Trophic Interactions and Lake Productivity
Carpenter, S. R., Kitchell, J. F., & Hodgson, J. R. (1985) · BioScience · doi:10.2307/1309989 · Cited by 2,646
Intervention Analysis with Applications to Economic and Environmental Problems
Box, G. E. P., & Tiao, G. C. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10480264 · Cited by 2,383
Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972) · Review of Educational Research · doi:10.3102/00346543042003237 · Cited by 1,826
Box, G. E. P. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728786 · Cited by 1,513
Environmental Impact Assessment: "Pseudoreplication" in Time?
Stewart-Oaten, A., Murdoch, W. W., & Parker, K. R. (1986) · Ecology · doi:10.2307/1939815 · Cited by 1,331
Self-citationAn Introduction to Probability Theory and Its Applications, Volume II
Grubbs, F. E., & Feller, W. (1967) · Technometrics · doi:10.2307/1266435 · Cited by 1,097
One Degree of Freedom for Non-Additivity
Tukey, J. W. (1949) · Biometrics · doi:10.2307/3001938 · Cited by 910
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478397 · Cited by 775
Maximum Likelihood Fitting of ARMA Models to Time Series With Missing Observations
Jones, R. H. (1980) · Technometrics · doi:10.1080/00401706.1980.10486171 · Cited by 612
Box, G. E. P. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728717 · Cited by 558
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.2307/2289131 · Cited by 548
The two-sample trimmed t for unequal population variances
YUEN, K. K. (1974) · Biometrika · doi:10.1093/biomet/61.1.165 · Cited by 486
Robertson, C. A., Andrews, D. F., Bickel, P. J., Hanpel, F. R., Huber, P. J., Rogers, W. H., & Tukey, J. W. (1972) · Biometrics · doi:10.2307/2528653 · Cited by 462
Interrupted Time Series Analysis
(2014) · Encyclopedia of Criminology and Criminal Justice · doi:10.1007/978-1-4614-5690-2_100362 · Cited by 385
The Large-Sample Power of Tests Based on Permutations of Observations
Hoeffding, W. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729436 · Cited by 365
Competition Among Refuging Sunfishes and Effects of Fish Density on Littoral Zone Invertebrates
Mittelbach, G. G. (1988) · Ecology · doi:10.2307/1941010 · Cited by 333
Concepts of Nonparametric Theory
Pratt, J. W., & Gibbons, J. D. (1981) · Springer series in statistics · doi:10.1007/978-1-4612-5931-2 · Cited by 317
Randomized Intervention Analysis and the Interpretation of Whole‐Ecosystem Experiments
Carpenter, S. R., Frost, T. M., Heisey, D., & Kratz, T. K. (1989) · Ecology · doi:10.2307/1941382 · Cited by 274
Effects of Body Size on the Predator‐Prey Interaction Between Pumpkinseed Sunfish and Gastropods
Osenberg, C. W., & Mittelbach, G. G. (1989) · Ecological Monographs · doi:10.2307/1943074 · Cited by 257
Self-citationA change in level of a non-stationary time series
Box, G. E. P., & Tiao, G. C. (1965) · Biometrika · doi:10.1093/biomet/52.1-2.181 · Cited by 242
How to Use the Two Sample t‐Test
Cressie, N. A. C., & Whitford, H. J. (1986) · Biometrical Journal · doi:10.1002/bimj.4710280202 · Cited by 216
Student's t-Test under Symmetry Conditions
Efron, B. (1969) · Journal of the American Statistical Association · doi:10.1080/01621459.1969.10501056 · Cited by 201
The Analysis of Transformed Data
Hinkley, D. V., & Runger, G. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478045 · Cited by 200
Large‐Scale Perturbations: Opportunities for Innovation
Carpenter, S. R. (1990) · Ecology · doi:10.2307/1938617 · Cited by 195
Trophic Relations and Ontogenetic Niche Shifts in Aquatic Ecosystems
Mittelbach, G. G., Osenberg, C. W., & Leibold, M. A. (1988) · Size-Structured Populations · doi:10.1007/978-3-642-74001-5_15 · Cited by 180
Self-citationOn the Behavior of Randomization Tests without a Group Invariance Assumption
Romano, J. P. (1990) · Journal of the American Statistical Association · doi:10.1080/01621459.1990.10474928 · Cited by 177
Osenberg, C. W., Werner, E. E., Mittelbach, G. G., & Hall, D. J. (1988) · Canadian Journal of Fisheries and Aquatic Sciences · doi:10.1139/f88-003 · Cited by 162
Self-citationConfidence Interval Robustness with Long-Tailed Symmetric Distributions
Gross, A. M. (1976) · Journal of the American Statistical Association · doi:10.1080/01621459.1976.10480359 · Cited by 153
Detecting Changes in Ecological Time Series
Jassby, A. D., & Powell, T. M. (1990) · Ecology · doi:10.2307/1938618 · Cited by 141
Replication and Treatment Strength in Whole‐Lake Experiments
Carpenter, S. R. (1989) · Ecology · doi:10.2307/1937550 · Cited by 139
A Note on the Selection of Data Transformations
Andrews, D. F. (1971) · Biometrika · doi:10.1093/biomet/58.2.249 · Cited by 120
On Obtaining Permutation Distributions in Polynomial Time
Pagano, M., & Tritchler, D. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477990 · Cited by 120
Robustness of MML estimators based on censored samples and robust test statistics
TIKU, M. L. (1980) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(80)90002-6 · Cited by 106
Bayesian Inference in Non‐Replicated Ecological Studies
Reckhow, K. H. (1990) · Ecology · doi:10.2307/1938619 · Cited by 102
The approximate behaviour and performance of the two-sample trimmed t
Yuen, K. K., & Dixon, W. J. (1973) · Biometrika · doi:10.1093/biomet/60.2.369 · Cited by 102
The robustness of the two—samplet—test over the Pearson system
Posten, H. O. (1978) · Journal of Statistical Computation and Simulation · doi:10.1080/00949657808810197 · Cited by 73
Playing Safe with Misweighted Means
Cressie, N. (1982) · Journal of the American Statistical Association · doi:10.1080/01621459.1982.10477882 · Cited by 17
Assessment of Thermal Discharges on Zooplankton in Conowingo Pond, Pennsylvania
Mathur, D., Robbins, T. W., & Purdy, E. J. (1980) · Canadian Journal of Fisheries and Aquatic Sciences · doi:10.1139/f80-123 · Cited by 16
Using biweight m-estimates in the two-sample problem part 1: symmetric populations
Kafadar, K. (1982) · Communication in Statistics- Theory and Methods · doi:10.1080/03610928208828358 · Cited by 15
Cited by
A Method for Assessing Hydrologic Alteration within Ecosystems
Richter, B. D., Baumgartner, J. V., Powell, J., & Braun, D. P. (1996) · Conservation Biology · doi:10.1046/j.1523-1739.1996.10041163.x · Cited by 2,463
Fundamentals of Microbial Community Resistance and Resilience
Shade, A., Peter, H., Allison, S. D., Baho, D. L., Berga, M., Bürgmann, H., Huber, D. H., Langenheder, S., Lennon, J. T., Martiny, J. B. H., et al. (2012) · Frontiers in Microbiology · doi:10.3389/fmicb.2012.00417 · Cited by 1,726
How much water does a river need?
RICHTER, B., BAUMGARTNER, J., WIGINGTON, R., & BRAUN, D. (1997) · Freshwater Biology · doi:10.1046/j.1365-2427.1997.00153.x · Cited by 1,601
Modelling and Quantitative Methods in Fisheries
Haddon, M. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/9781439894170 · Cited by 1,040
Defining and measuring river health
Karr, J. R. (1999) · Freshwater Biology · doi:10.1046/j.1365-2427.1999.00427.x · Cited by 675
Metadata
- Authors
- Stewart-Oaten, A., Bence, J. R., & Osenberg, C. W.
- Venue
- Ecology
- Publisher
- Wiley
- Published
- 1992-08
- Volume
- 73
- Issue
- 4
- Pages
- 1396-1404
- ISSN
- 0012-9658 (ISSN-L)
- 1939-9170
- OpenAlex
- W2126549349
- Cited by
- 269
- References
- 44
- FWCI
- 8.54
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.