Pseudoreplication is (still) a problem.
Abstract
Hurlbert (1984) laid out a rationale for why experimental designs need to consider statistical problems related to pseudoreplication of data. This paper has been a cornerstone for more proper experimental design in many fields. Nonetheless, Schank and Koehnle (2009) argue that pseudoreplication is no longer a relevant issue. We disagree with this conclusion. We list a number of distortions of the original Hurlbert paper that are found in Schank and Koehnle. In addition, we note that some of the original suggestions in Hurlbert are no longer valid because statistical techniques have inevitably evolved to provide stronger tests than were available 25 years ago.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pseudoreplication has been addressed repeatedly in ecology, agriculture, psychology, and neuroscience and acknowledged as one of the most common statistical mistakes in scientific literature.
“Pseudoreplication has been addressed repeatedly in ecology, agriculture, psychology, and neuroscience and acknowledged as one of the most common statistical mistakes in scientific literature4–9.”
From A practical solution to pseudoreplication bias in single-cell studies (2021) · cites this paper as [7]
Cites
Biometry: The Principles and Practice of Statistics in Biological Research
Sokal, R. R., & Rohlf, F. J. (1969) · Cited by 20,963 · see citing works
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
Cohen, P., Cohen, P., West, S. G., & Aiken, L. S. (2014) · Psychology Press eBooks · doi:10.4324/9781410606266 · Cited by 20,819 · see citing works
Pseudoreplication and the Design of Ecological Field Experiments
Hurlbert, S. H. (1984) · Ecological Monographs · doi:10.2307/1942661 · Cited by 8,377 · see citing works
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249 · see citing works
(2006) · Cited by 3,705 · see citing works
A Primer of Ecological Statistics
Gotelli, N. J., & Ellison, A. M. (2004) · Cited by 1,237 · see citing works
Suggested experimental designs for song playbacks
Kroodsma, D. E. (1989) · Animal Behaviour · doi:10.1016/0003-3472(89)90039-0 · Cited by 421 · see citing works
Pseudoreplication in playback experiments, revisited a decade later
Kroodsma, D. E., Byers, B. E., Goodale, E., Johnson, S., & Liu, W. (2001) · Animal Behaviour · doi:10.1006/anbe.2000.1676 · Cited by 387 · see citing works
Machlis, L. E., Dodd, P. W. D., & Fentress, J. C. (1985) · Zeitschrift für Tierpsychologie · doi:10.1111/j.1439-0310.1985.tb00124.x · Cited by 331 · see citing works
On misinterpretations of pseudoreplication and related matters: a reply to Oksanen
Hurlbert, S. H. (2004) · Oikos · doi:10.1111/j.0030-1299.2004.12752.x · Cited by 149 · see citing works
Pseudoreplication is a pseudoproblem.
Schank, J. C., & Koehnle, T. J. (2009) · Journal of comparative psychology · doi:10.1037/a0013579 · Cited by 146 · see citing works
Is there an ideal behavioural experiment?
Wiley, R. H. (2003) · Animal Behaviour · doi:10.1006/anbe.2003.2231 · Cited by 74 · see citing works
Freed-Brown, S. G., King, A. P., Miller, J., & West, M. J. (2006) · Behaviour · doi:10.1163/156853906778987533 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Linear models and linear mixed effects models in R with linguistic applicationsPreprint
Winter, B. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.5499 · Cited by 656 · see citing works
A practical solution to pseudoreplication bias in single-cell studies
Zimmerman, K. D., Espeland, M. A., & Langefeld, C. D. (2021) · Nature Communications · doi:10.1038/s41467-021-21038-1 · Cited by 362 · see citing works
Davies, G. M., & Gray, A. (2015) · Ecology and Evolution · doi:10.1002/ece3.1782 · Cited by 333 · see citing works
Studying synapses in human brain with array tomography and electron microscopy
Kay, K. R., Smith, C., Wright, A. K., Serrano‐Pozo, A., Pooler, A. M., Koffie, R. M., Bastin, M. E., Bak, T. H., Abrahams, S., Kopeikina, K. J., et al. (2013) · Nature Protocols · doi:10.1038/nprot.2013.078 · Cited by 114 · see citing works
Pseudoreplication: a widespread problem in primate communication research
Waller, B. M., Warmelink, L., Liebal, K., Micheletta, J., & Slocombe, K. E. (2013) · Animal Behaviour · doi:10.1016/j.anbehav.2013.05.038 · Cited by 68 · see citing works
Reinventing Biostatistics Education for Basic Scientists
Weissgerber, T. L., Garovic, V. D., Milin-Lazovic, J. S., Winham, S. J., Zoran Obradovic, A., Trzeciakowski, J. P., & Milic, N. M. (2016) · PLoS Biology · doi:10.1371/journal.pbio.1002430 · Cited by 68 · see citing works
PSEUDOREPLICATION IN PHONETIC RESEARCH
Winter, B. (2011) · ICPhS · Cited by 54 · see citing works
Brown, H. C. A., Berninger, F. A., Larjavaara, M., & Appiah, M. (2020) · Forest Ecology and Management · doi:10.1016/j.foreco.2020.118236 · Cited by 52 · see citing works
Scents and sensibility: Best practice in insect olfactometer bioassays
Roberts, J. M., Clunie, B., Leather, S. R., Harris, W. E., & Pope, T. W. (2023) · Entomologia Experimentalis et Applicata · doi:10.1111/eea.13351 · Cited by 53 · see citing works
Using meta-analysis for evidence synthesis: The case of incomplete neutralization in German
Nicenboim, B., Roettger, T. B., & Vasishth, S. (2018) · Journal of Phonetics · doi:10.1016/j.wocn.2018.06.001 · Cited by 44 · see citing works
Loss, C. M., Melleu, F. F., Domingues, K., Lino-de-Oliveira, C., & Viola, G. G. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.763428 · Cited by 44 · see citing works
What stabilizes biological soil crusts in the Negev Desert?
Felde, V. J. M. N. L., Chamizo, S., Felix‐Henningsen, P., & Drahorad, S. L. (2017) · Plant and Soil · doi:10.1007/s11104-017-3459-7 · Cited by 41 · see citing works
Links between slope aspect and rate of litter decomposition on inland dunes
Jasińska, J., Sewerniak, P., & Markiewicz, M. (2018) · CATENA · doi:10.1016/j.catena.2018.09.025 · Cited by 40 · see citing works
Male vocal competition is dynamic and strongly affected by social contexts in music frogs
Fang, G., Jiang, F., Yang, P., Cui, J., Brauth, S. E., & Tang, Y. (2013) · Animal Cognition · doi:10.1007/s10071-013-0680-5 · Cited by 30 · see citing works
Al-Tally, H. A. (2014) · Victoria University Research Repository (Victoria University) · Cited by 21 · see citing works
Improving rigor and reproducibility in western blot experiments with the blotRig analysis
Omondi, C., Chou, A., Fond, K. A., Morioka, K., Joseph, N. R., Sacramento, J. A., Iorio, E., Espin, A. T., Radabaugh, H. L., Davis, J. A., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-70096-0 · Cited by 19 · see citing works
Methodological issues when using face prototypes: A case study on the Faceaurus dataset
Bovet, J., Tognetti, A., & Pollet, T. V. (2022) · Evolutionary Human Sciences · doi:10.1017/ehs.2022.25 · Cited by 14 · see citing works
Pseudoreplication in use of predator stimuli in experiments on antipredator responses
Johnson, W. T., & Freeberg, T. M. (2016) · Animal Behaviour · doi:10.1016/j.anbehav.2016.07.006 · Cited by 15 · see citing works
Self-citationShorebird embryos exhibit anti‐predator responses
Kostoglou, K. N., van Dongen, W. F. D., Bowe, S. J., & Weston, M. A. (2021) · Ibis · doi:10.1111/ibi.12969 · Cited by 14 · see citing works
Fan, Y., Fang, K., Sun, R., Shen, D., Yang, J., Tang, Y., & Fang, G. (2021) · Current Zoology · doi:10.1093/cz/zoab085 · Cited by 13 · see citing works
Local–foreign dialect discrimination and responses to mixed-dialect duets in the North Island kōkako
Bradley, D. W., Molles, L. E., & Waas, J. R. (2012) · Behavioral Ecology · doi:10.1093/beheco/ars201 · Cited by 12 · see citing works
Grade-control outdoor turning flight of robo-pigeon with quantitative stimulus parameters
Fang, K., Mei, H., Tang, Y., Wang, W., Wang, H., Wang, Z., & Dai, Z. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1143601 · Cited by 9 · see citing works
Auditory perception exhibits sexual dimorphism and left telencephalic dominance inXenopus laevis
Fan, Y., Yue, X., Xue, F., Cui, J., Brauth, S. E., Tang, Y., & Fang, G. (2018) · Biology Open · doi:10.1242/bio.035956 · Cited by 9 · see citing works
Using meta-analysis for evidence synthesis: The case of incomplete neutralization in GermanPreprint
Nicenboim, B., Roettger, T. B., & Vasishth, S. (2018) · doi:10.31234/osf.io/p5a4z · Cited by 9 · see citing works
Pseudoreplication in Primate Communication Research: 10 Years On
Whitehouse, J., Clark, P. R., Micheletta, J., Liebal, K., Slocombe, K. E., & Waller, B. M. (2023) · International Journal of Primatology · doi:10.1007/s10764-023-00399-y · Cited by 7 · see citing works
Non‐Redundant Social Information Use in Avian Flocks with Multisensory Stimuli
Randolet, J., Lucas, J. R., & Fernández‐Juricic, E. (2014) · Ethology · doi:10.1111/eth.12212 · Cited by 6 · see citing works
Self-citationFang, K., Tang, Y., Zhang, B., & Fang, G. (2022) · Communications Biology · doi:10.1038/s42003-022-03504-8 · Cited by 4 · see citing works
Pseudoreplication bias in single-cell studies; a practical solution
Zimmerman, K. D., Espeland, M. A., & Langefeld, C. D. (2020) · doi:10.1101/2020.01.15.906248 · Cited by 332 · see citing works
Kostoglou, K. N., van Dongen, W. F. D., & Weston, M. A. (2021) · Ibis · doi:10.1111/ibi.12986 · Cited by 3 · see citing works
Martinez, A. S., Underwood, A. J., Christofoletti, R. A., Pardal, A., Fortuna, M. D., Silva, J., Morais, G. C., & Lana, P. D. C. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.155229 · Cited by 3 · see citing works
Snyder, S. W., Meyer, W. C., Jorgenson, S. L., Contreras, E. B., Bland, T., & Cresko, W. A. (2026) · Journal of Evolutionary Biology · doi:10.1093/jeb/voag024 · Cited by 2 · see citing works
Candy, S. G., & Englefield, B. (2022) · Current Journal of Applied Science and Technology · doi:10.9734/cjast/2022/v41i333946 · Cited by 2 · see citing works
Dynamically Controlled Flight Altitudes in Robo-Pigeons via Locus Coeruleus Neurostimulation
Fang, K., Wang, Z., Tang, Y., Guo, X., Li, X., Wang, W., Liu, B., & Dai, Z. (2025) · Research · doi:10.34133/research.0632 · Cited by 2 · see citing works
Replication vs. Pseudoreplication: Are We Making Too Big a Deal of This?
Wiersma, Y. F. (2022) · Landscape series · doi:10.1007/978-3-030-95189-4_4 · Cited by 2 · see citing works
The movement and positioning of soccer officials in relation to their decision making
Hazmi, A. A. (2016) · Cited by 2 · see citing works
Qing, T., Shen, D., Fan, Y., Shi, S., Li, L., Wu, Y., & Fang, G. (2025) · Communications Biology · doi:10.1038/s42003-025-08170-0 · Cited by 1 · see citing works
Niu, W., Shen, D., Sun, R., Fan, Y., Yang, J., Zhang, B., & Fang, G. (2022) · Biology · doi:10.3390/biology11060879 · Cited by 1 · see citing works
Building organizational capacity framework applied to academic spin-off creation
Sarazen, J. (2020) · doi:10.17760/d20350268 · Cited by 1 · see citing works
- Authors
- Freeberg, T. M., & Lucas, J. R.
- Affiliations
- University of Tennessee at Knoxville (US)
- Venue
- Journal of Comparative Psychology
- Published
- 2009-11
- Volume
- 123
- Issue
- 4
- Pages
- 450-451
- ISSN
- 0093-4127 (ISSN-L)
- 1939-2087
- 0735-7036
- OpenAlex
- W1990387061
- Cited by
- 37
- References
- 13
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 1 · 2023: 3 · 2022: 6 · 2021: 5 · 2020: 2 · 2018: 4 · 2017: 1 · 2016: 3 · 2015: 1 · 2014: 2 · 2013: 4 · 2012: 1
- FWCI
- 0.55
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of comparative psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- pseudoreplication, experimental design, statistical power, repeated measures, within-subjects design
- MeSH terms
- Analysis of Variance, Animals, Behavior, Animal, Ecology, Humans, Research Design, Reproducibility of Results, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review