Pseudoreplication and the Design of Ecological Field Experiments
The DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index.
Abstract
Pseudoreplication is defined as the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent. In ANOVA terminology, it is the testing for treatment effects with an error term inappropriate to the hypothesis being considered. Scrutiny of 176 experimental studies published between 1960 and the present revealed that pseudoreplication occurred in 27% of them, or 48% of all such studies that applied inferential statistics. The incidence of pseudoreplication is especially high in studies of marine benthos and small mammals. The critical features of controlled experimentation are reviewed. Nondemonic intrusion is defined as the impingement of chance events on an experiment in progress. As a safeguard against both it and preexisting gradients, interspersion of treatments is argued to be an obligatory feature of good design. Especially in small experiments, adequate interspersion can sometimes be assured only by dispensing with strict randomization procedures. Comprehension of this conflict between interspersion and randomization is aided by distinguishing pre—layout (or conventional) and layout—specific alpha (probability of type I error). Suggestions are offered to statisticians and editors of ecological journals as to how ecologists' understanding of experimental design and statistics might be improved.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Using countries as units in statistical tests may therefore lead to pseudo-replication, inflated sample size and misleading results as regards probability levels [50, 51]”
From Human fertility in relation to education, economy, religion, contraception, and family planning programs (2020) · cites this paper as [50]
In the citing paper
“countries may not be statistically independent units.”
From Human fertility in relation to education, economy, religion, contraception, and family planning programs (2020) · cites this paper as [50]
In the citing paper
“The tautological structure of the Kauffman and Cummings model invalidates Ripple et al.'s analysis because applying it violates the scientific principle that conclusions must be supported by evidence that is independent of the data used to generate them (Hurlbert 1984; Nakagawa & Parker 2015).”
From Flawed analysis invalidates claim of a strong Yellowstone trophic cascade after wolf reintroduction: A comment on Ripple et al. (2025) (2025) · cites this paper as Hurlbert 1984
In the citing paper
“A reason for this small number is that many studies incorporated experimental designs that suffered from pseudoreplication [79].”
From Land use change to bioenergy: A meta-analysis of soil carbon and GHG emissions (2015) · cites this paper as [79]
In the citing paper
“Pseudoreplication leads to a biased and incorrect inference, and it should be carefully considered (Hurlbert, 1984).”
From Acoustic indices as proxies for biodiversity: a meta‐analysis (2022) · cites this paper as Hurlbert, 1984
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 26,755
Principles and procedures of statistics.
Steel, R. G. D., & Torrie, J. H. (1960) · Cited by 24,135
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Principles and procedures of statistics: McGraw-Hill Book Company, Inc. New York Toronto London.
Steel, R., & Torrie, J. H. (1960) · Cited by 4,633
Food Webs: Linkage, Interaction Strength and Community Infrastructure
Paine, R. T. (1980) · The Journal of Animal Ecology · doi:10.2307/4220 · Cited by 1,824
Sampling Design and Statistical Methods for Environmental Biologists
Green, R. H. (1979) · Andalas University Repository (Andalas University) · Cited by 1,550
Biogeochemistry of a forested ecosystem
(2014) · Choice Reviews Online · doi:10.5860/choice.51-6175 · Cited by 1,550
Design and Analysis of Experiments
(2007) · Modern Engineering Statistics · doi:10.1002/9780470187548.ch12 · Cited by 1,424
Paine, R. T. (1974) · Oecologia · doi:10.1007/bf00345739 · Cited by 1,403
Cited by
Items 1,001–1,029 of 1,029
Woody Debris, Forest-Stream Succession, and Catchment Geomorphology
Trotter, E. H. (1990) · Journal of the North American Benthological Society · doi:10.2307/1467447 · Cited by 103
Facial correlates of emotional behaviour in the domestic cat (Felis catus)
Bennett, V. A., Gourkow, N., & Mills, D. S. (2017) · Behavioural Processes · doi:10.1016/j.beproc.2017.03.011 · Cited by 113
Fitak, R. R., & Johnsen, S. (2017) · Journal of Experimental Biology · doi:10.1242/jeb.167056 · Cited by 92
“The most widely criticized mistakes include pseudoreplication (Hurlbert, 1984), failure to correct for multiple testing (Noble, 2009; Pike, 2011; Verhoeven et al., 2005) and low statistical power (Jennions and Møller, 2003; Nakagawa, 2004).”
BackgroundFuel treatments and fire severity: A meta-analysis
Martinson, E. J., & Omi, P. N. (2013) · doi:10.2737/rmrs-rp-103 · Cited by 91
“However, field studies in ecology (Adams and others 1997) and especially in wildland fire research (Van Mantgem and others 2001) are more often observational than experimental, rarely conform to parametric assumptions, and are generally pseudoreplicated (in the sense of Hurlbert 1984).”
BackgroundExperimental trampling and vegetation recovery in some forest and heathland communities
Roovers, P., Verheyen, K., Hermy, M., & Gulinck, H. (2004) · Applied Vegetation Science · doi:10.1111/j.1654-109x.2004.tb00601.x · Cited by 84
Denny, S. K., & Fanning, L. M. (2016) · International Indigenous Policy Journal · doi:10.18584/iipj.2016.7.3.4 · Cited by 77
“Knowledge about the resource is science-based using a process that follows a systematic method to test questions from observations and is designed to reduce bias (Hurlbert, 1984).”
MethodKain, M., Battaglia, L., Royo, A., & Carson, W. P. (2011) · The Journal of the Torrey Botanical Society · doi:10.3159/torrey-d-11-00018.1 · Cited by 71
“The fence was not replicated and we used subpots (n 5 7) as replicates (pseudoreplicates sensu Hurlbert 1984); thus, our results should be interpreted with caution.”
MethodAn assessment of statistical methods for nonindependent data in ecological meta‐analyses
Song, C., Peacor, S. D., Osenberg, C. W., & Bence, J. R. (2020) · Ecology · doi:10.1002/ecy.3184 · Cited by 71
“This type of nonindependence has been called pseudoreplication and can seriously undermine statistical inferences (Hurlbert 1984).”
BackgroundEcosystem services across the aquatic–terrestrial boundary: Linking ponds to pollination
Stewart, R. I. A., Andersson, G. K. S., Brönmark, C., Klatt, B. K., Hansson, L.-A., Zülsdorff, V., & Smith, H. G. (2017) · Basic and Applied Ecology · doi:10.1016/j.baae.2016.09.006 · Cited by 67
Hummingbirds pay a high cost for a warm drink
Lotz, C. N., Mart�nez del Rio, C., & Nicolson, S. W. (2003) · Journal of Comparative Physiology B: Biochemical, Systemic, and Environmental Physiology · doi:10.1007/s00360-003-0346-8 · Cited by 66
“We used individual birds as components in these models, to avoid pseudoreplication (Hurlbert 1984).”
MethodFaget, M., Blossfeld, S., Gillhaussen, P. V., Schurr, U., & Temperton, V. M. (2013) · Frontiers in Plant Science · doi:10.3389/fpls.2013.00392 · Cited by 63
“…to note that the spatial scale of a planar optode ROI is very much smaller than that of the whole rhizotron, such that we considered the ROIs as replicates in most cases [e.g., see Figure 5; see arguments in Hurlbert (1984) on the issue of spatial scale and pseudo-replication in experiments].”
BackgroundThomsen, M., Wernberg, T., Olden, J., Byers, J. E., Bruno, J., Silliman, B., & Schiel, D. (2014) · NeoBiota · doi:10.3897/neobiota.22.6224 · Cited by 51
“…and determined whether the experimental design was spatially pseudo-replicated (i.e. with only one ‘spatially independent’ control and/or treatment, Hurlbert 1984), and if the study reported an average invasion impact on resident organisms with an associated measure of Mads S. Thomsen et al. /…”
BackgroundResultUse of meta-analysis in forest biodiversity research: key challenges and considerations
Spake, R., & Doncaster, C. P. (2017) · Forest Ecology and Management · doi:10.1016/j.foreco.2017.05.059 · Cited by 60
“Hurlbert (1984) distinguished ‘acceptable’ study designs, including completely randomized, randomized block and systematic designs all with independently interspersed treatment replicates, from pseudoreplicated designs, which lack independence across replicates (Fig.”
BackgroundDewar, A. E., Thomas, J. L., Scott, T. W., Wild, G., Griffin, A. S., West, S. A., & Ghoul, M. (2021) · Nature Ecology & Evolution · doi:10.1038/s41559-021-01573-2 · Cited by 44
Angel, P. N., Graves, D. H., Barton, C., Warner, R. C., Conrad, P. W., Sweigard, R. J., & Agouridis, C. (2006) · Journal American Society of Mining and Reclamation · doi:10.21000/jasmr06020045 · Cited by 41
“Although the study design was pseudo-replicated (Hurlburt 1984) by block, testing the significance of treatments was possible.”
MethodPartial migration or just habitat selection? Seasonal movements of roe deer in an Alpine population
Gaudry, W., Saïd, S., Gaillard, J.-M., Chevrier, T., Loison, A., Maillard, D., & Bonenfant, C. (2015) · Journal of Mammalogy · doi:10.1093/jmammal/gyv055 · Cited by 38
Pseudoreplication is (still) a problem.
Freeberg, T. M., & Lucas, J. R. (2009) · Journal of Comparative Psychology · doi:10.1037/a0017031 · Cited by 37
“We are not convinced that Hurlbert’s argument for researchers to work toward minimizing confounds and maximizing statistical independence of their data points is unsound, nor that Hurlbert sets impossible standards for researchers. More planning, more effort at adequate sampling, and more care, yes; impossible, no. As…”
BackgroundInfluentialLim, H. S., Fraser, A., & Knights, A. M. (2020) · Environmental Research Letters · doi:10.1088/1748-9326/ab83ae · Cited by 36
“As such, data from each image are therefore both pseudo-replicated (Hurlbert 1984) and autocorrelated making formal hypothesis-driven statistical tests inappropriate, not least due to the risk of TypeII error.”
Giles, A., Fanning, L., Denny, S., & Paul, T. (2016) · Human Ecology · doi:10.1007/s10745-016-9814-0 · Cited by 35
“The scientific way of knowing, also referred to as sciencebased knowledge (Kuhn 2012), is a systematic verification process to test questions from observations and designed to reduce bias (Hurlbert 1984).”
BackgroundApplication of the hierarchical bootstrap to multi-level data in neuroscience
Saravanan, V., Berman, G. J., & Sober, S. J. (2020) · arXiv · doi:10.48550/arxiv.2007.07797 · Cited by 26
“While pseudoreplication was first extensively reported on in ecological studies [5, 48], it has since been identified as a common problem in other fields, including neuroscience [6].”
BackgroundResultiMyoblasts for ex vivo and in vivo investigations of human myogenesis and disease modeling
Guo, D., Daman, K., Chen, J. J., Shi, M.-J., Yan, J., Matijasevic, Z., Rickard, A. M., Bennett, M. H., Kiselyov, A., Zhou, H., et al. (2022) · eLife · doi:10.7554/elife.70341 · Cited by 26
“This analysis was based on pseudo- bulk expression profiles (Tung et al., 2017) derived from scRNA- Seq data by summing counts of all cells from the same cell class and donor to avoid spurious results due to pseudoreplication (Hurlbert, 1984).”
MethodSmith, D. S., Fettig, S. M., & Bowker, M. A. (2016) · Forest Ecology and Management · doi:10.1016/j.foreco.2015.11.020 · Cited by 24
Social significance of trunk use in captive Asian elephants
Yasui, S., & Idani, G. (2017) · Ethology Ecology & Evolution · doi:10.1080/03949370.2016.1179684 · Cited by 18
“We also included animal identification (ID) as a random effect to avoid pseudo-replication (Hurlbert 1984).”
MethodPeteiro, C., Bidegain, G., & Sánchez, N. (2019) · Algal Research · doi:10.1016/j.algal.2019.101678 · Cited by 17
Smith, E. P. (2012) · Encyclopedia of Environmetrics · doi:10.1002/9780470057339.vab001.pub2 · Cited by 16
Association between taxonomic relatedness and interspecific mortality in captive ungulates
Hanzlíková, V., Pluháček, J., & Čulík, L. (2014) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2014.01.010 · Cited by 5
Pseudoreplication bias in single-cell studies; a practical solution
Zimmerman, K. D., Espeland, M. A., & Langefeld, C. D. (2020) · doi:10.1101/2020.01.15.906248 · Cited by 332
MacNulty, D. R., Cooper, D., Procko, M., & Clark-Wolf, T. J. (2025) · Global Ecology and Conservation · doi:10.1016/j.gecco.2025.e03899 · Cited by 1
“The tautological structure of the Kauffman and Cummings model invalidates Ripple et al. ’ s analysis because applying it violates the scientific principle that conclusions must be supported by evidence that is independent of the data used to generate them (Hurlbert, 1984; Nakagawa and Parker, 2015).”
MacNulty, D., Cooper, D., Procko, M., & Clark-Wolf, T. (2025) · doi:10.32942/x2qq0c
Metadata
- Authors
- Hurlbert, S. H.
- Corresponding author
- Hurlbert, Stuart H.
- Venue
- Ecological Monographs
- Publisher
- Wiley
- Published
- 1984-06
- Volume
- 54
- Issue
- 2
- Pages
- 187-211
- ISSN
- 0012-9615 (ISSN-L)
- 1557-7015
- OpenAlex
- W2139156166
- Cited by
- 8,377
- References
- 102
- Citations by year
- 2026: 283 · 2025: 212 · 2024: 155 · 2023: 167 · 2022: 164 · 2021: 212 · 2020: 221 · 2019: 233 · 2018: 231 · 2017: 226 · 2016: 266 · 2015: 279 · 2014: 296 · 2013: 280 · 2012: 254
- FWCI
- 120.39
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Ecological Monographs (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- pseudoreplication, experimental design, randomization, type I error, marine benthos, small mammals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.