Cited by 381Statistical Methods and Bayesian InferenceMeta-analysis and systematic reviewsStatistical Methods in Epidemiology

The DOI matches its Crossref record; 92 of the 92 references it cites were found in a registry or an index.

The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?

Open access

Lazic, Stanley E·2010·BMC Neuroscience

DOI 10.1186/1471-2202-11-5PMC PMC2817684

Open this paper in CoChat — free account Open PDF

Abstract

BACKGROUND: Pseudoreplication occurs when observations are not statistically independent, but treated as if they are. This can occur when there are multiple observations on the same subjects, when samples are nested or hierarchically organised, or when measurements are correlated in time or space. Analysis of such data without taking these dependencies into account can lead to meaningless results, and examples can easily be found in the neuroscience literature. RESULTS: A single issue of Nature Neuroscience provided a number of examples and is used as a case study to highlight how pseudoreplication arises in neuroscientific studies, why the analyses in these papers are incorrect, and appropriate analytical methods are provided. 12% of papers had pseudoreplication and a further 36% were suspected of having pseudoreplication, but it was not possible to determine for certain because insufficient information was provided. CONCLUSIONS: Pseudoreplication can undermine the conclusions of a statistical analysis, and it would be easier to detect if the sample size, degrees of freedom, the test statistic, and precise p-values are reported. This information should be a requirement for all publications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1186/1471-2202-11-5

Cites

92 of 92 references verified0 citation errors · 0 not checkable

Statistics

Figure 1 An example of pseudoreplication. Two rats are sampled from a population with a mean (μ) of 50 and a standard deviation (σ) of 10, and ten measurements of an arbitrary outcome variable are made on each rat. The first (incorrect) 90% CI uses all 20 data points and does not account for the hierarchical nature of the data. For the second 90% CI, the mean of the ten values for each rat are calculated first, and then only these two averaged values are used for the calculation of the CI. The error bar on the left is incorrect because each of the 20 data points are not a random sample from the whole population, but rather samples within two rats. This is evident from the fact that the 10 points are normally distributed around the mean of their respective rats, but not normally distributed around the population mean (horizontal grey line), as would be expected when independent samples are randomly drawn from a population. Increasing the number of observations on each rat does not lead to a more precise estimate of μ, which requires more rats. Note that 90% CI are plotted for clarity because the graph needs to be greatly compressed to display the 95% CI.CC BY · PubMed Central

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Mixed effects models and extensions in ecology with R
Zuur, A. F., Ieno, E. N., Walker, N., Saveliev, A. A., & Smith, G. M. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-87458-6 · Cited by 17,963
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
Gentleman R: R: A language for data analysis and graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,747
Mixed-Effects Models in Sand S-PLUS
Pinheiro, J. C., & Bates, D. M. (2000) · Statistics and Computing · doi:10.1007/978-1-4419-0318-1 · Cited by 11,187
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Mixed-Effects Models in S and S-PLUS
(2001) · Technometrics · doi:10.1198/tech.2001.s574 · Cited by 9,278
Pseudoreplication and the Design of Ecological Field Experiments
Hurlbert, S. H. (1984) · Ecological Monographs · doi:10.2307/1942661 · Cited by 8,377
The R Book
Crawley, M. J. (2007) · Wiley eBooks · doi:10.1002/9780470515075 · Cited by 6,980
The R Book
Crawley, M. J. (2007) · Medical Entomology and Zoology · Cited by 5,431

Show all 90 references

Sorted by citations

Items 101–200 of 381

Metabolomics Peak Analysis Computational Tool (MPACT): An Advanced Informatics Tool for Metabolomics and Data Visualization of Molecules from Complex Biological Samples
Samples, R. M., Puckett, S. P., & Balunas, M. J. (2023) · Analytical Chemistry · doi:10.1021/acs.analchem.2c04632 · Cited by 24
Pseudoreplication and the usage of biomarkers in ecotoxicological bioassays
Osório, F. H. T., Galvan, G. L., Marques, A. E. M. L., Santos, G. S., Pereira, L. S., da Silva, T. A., de Assis, H. C. S., Barbosa, R. V., & Cestari, M. M. (2017) · Environmental Toxicology and Chemistry · doi:10.1002/etc.3823 · Cited by 22
The First Call Note Plays a Crucial Role in Frog Vocal Communication
Yue, X., Fan, Y., Xue, F., Brauth, S. E., Tang, Y., & Fang, G. (2017) · Scientific Reports · doi:10.1038/s41598-017-09870-2 · Cited by 22
A metabolomic study on the gender-dependent effects of maternal exposure to fenvalerate on neurodevelopment in offspring mice
Zhang, H., Lu, T., Feng, Y., Sun, X. L., Xu, Y., Zhou, K., Sun, R., Wang, Y., Wang, X., & Chen, M. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.136130 · Cited by 22
Normalization and Statistical Analysis of Multiplexed Bead-based Immunoassay Data Using Mixed-effects Modeling
Clarke, D. C., Morris, M. K., & Lauffenburger, D. A. (2012) · Molecular & Cellular Proteomics · doi:10.1074/mcp.m112.018655 · Cited by 22
“1C) (44).”
Avoiding false discoveries in single-cell RNA-seq by revisiting the first Alzheimer’s disease dataset
Murphy, A. E., Fancy, N. N., & Skene, N. (2023) · eLife · doi:10.7554/elife.90214 · Cited by 25
“This point is the major issue with pseudoreplication approaches which overestimate confidence when performing DE due to the statistical dependence between cells from the same patient not being considered ( Squair et al., 2021 ; Lazic, 2010 ).”
Background
Increased alcohol consumption in sleep-restricted rats is mediated by delta FosB induction
García‐García, F., Priego-Fernández, S., López-Muciño, L. A., Acosta-Hernández, M. E., & Peña-Escudero, C. (2021) · Alcohol · doi:10.1016/j.alcohol.2021.02.004 · Cited by 21
Implementing Performance Accommodation Mechanisms in Online BCI for Stroke Rehabilitation: A Study on Perceived Control and Frustration
Jochumsen, M., Hougaard, B. I., Kristensen, M. S., & Knoche, H. O. (2022) · Sensors · doi:10.3390/s22239051 · Cited by 21
“To counter potential pseudoreplication [51] from our repeated measures design, we used Participant as the basis for the null model and modeled it as random intercepts to account for by-subject baseline rating differences.”
Method
Enriched environment restricted to gestation accelerates the development of sensory and motor circuits in the rat pup
Cárdenas, L., García‐García, F., Roque, I. S., Martínez, A. J., Coria‐Ávila, G. A., & Corona-Morales, A. A. (2015) · International Journal of Developmental Neuroscience · doi:10.1016/j.ijdevneu.2014.11.008 · Cited by 21
“To address effects of pseudoreplication appropriately, data of each outcome variable from pups within the same litter were averaged and we used the litter as the unit of measure for statistical comparisons (Lazic, 2010).”
Method
Field Evaluation of Predacious Mites (Acari: Phytoseiidae) for Biological Control of Citrus Red Mite,Panonychus citri(Trombidiformes: Tetranychidae)
Fadamiro, H. Y., Akotsen‐Mensah, C., Xiao, Y., & Anikwe, J. C. (2013) · Florida Entomologist · doi:10.1653/024.096.0111 · Cited by 21
The effects of mining machinery noise of different frequencies on the behaviour, faecal corticosterone and tissue morphology of wild mice ( Mus musculus )
Mancera, K. F., Lisle, A. T., Allavena, R., & Phillips, C. J. C. (2017) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2017.08.008 · Cited by 21
Tuning metaheuristics by sequential optimisation of regression models
Trindade, Á. R., & Campelo, F. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105829 · Cited by 21
“Failure to account for this particular fact would result in pseudoreplication [18, 26], a violation of the assumption of independence underlying the statistical approaches used in most tuning procedures that leads to inflated type-I errors in inferential tests, and to artificially reduced standard errors in descriptiv…”
Background
Samples in many cell-based experiments are matched/paired but taking this into account does not always increase power of statistical tests for differences in means
Zweifach, A. (2023) · Molecular Biology of the Cell · doi:10.1091/mbc.e23-05-0159 · Cited by 20
“…type 1 error rate when applying Student’s t test to normalized data to ∼ 6% (see Supplemental data file “Data_Fig3A_B_C.csv”)–this is one of the few statistical errors apart from pseudoreplication (Lazic, 2010; Eisner, 2021) that can increase the FPR, so the strategy was not investigated further.”
Method
Effects of Freshly Irradiated vs Irradiated and Stored Red Blood Cell Transfusion on Cerebral Oxygenation in Preterm Infants
Saito‐Benz, M., Bennington, K., Gray, C. L., Murphy, W. G., Flanagan, P., Steiner, F., Atkinson, G., & Berry, M. J. (2022) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2022.0152 · Cited by 20
Quantifying the Behavioural Relevance of Hippocampal Neurogenesis
Lazic, S. E., Fuß, J., & Gass, P. (2014) · PLoS ONE · doi:10.1371/journal.pone.0113855 · Cited by 20
“This can be determined by inconsistencies in numbers of data points across relevant figures, a discrepancy between the number of animals reported in the methods compared with those shown in the figures, or when the reported degrees of freedom do not match the indicated sample size and method of analysis [19].”
BackgroundResultSelf-citation
Unit of analysis issues in laboratory-based research
Parsons, N., Teare, M. D., & Sitch, A. J. (2018) · eLife · doi:10.7554/elife.32486 · Cited by 19
“…generally for reported analyses in bioscience and medicine (Aarts et al., 2014; Altman and Bland, 1997; Bunce et al., 2014; Fleming et al., 2013; Lazic, 2010; Calhoun et al., 2008; Divine et al., 1992), and also feed into widely acknowledged issues around the lack of reproducibility and…”
Background
Exposure of laboratory animals to small air ions: a systematic review of biological and behavioral studies
Bailey, W. H., Williams, A. L., & Leonhard, M. J. (2018) · BioMedical Engineering OnLine · doi:10.1186/s12938-018-0499-z · Cited by 21
Decoding cognitive concepts from neuroimaging data using multivariate pattern analysis
Alizadeh, S., Jamalabadi, H., Schönauer, M., Leibold, C., & Gais, S. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.058 · Cited by 20
“Although such considerations are not new for classical statistics (Galbraith et al., 2010; Lazic, 2010; Aarts et al., 2014, 2015; Moen et al., 2016), the implications for the use of MVPA must be further explored.”
Background
Changes in Posttraumatic Brain Edema in Craniectomy-Selective Brain Hypothermia Model Are Associated With Modulation of Aquaporin-4 Level
Szczygielski, J., Glameanu, C., Müller, A., Klotz, M., Sippl, C., Hubertus, V., Schäfer, K., Mautes, A. E. M., Schwerdtfeger, K., & Oertel, J. (2018) · Frontiers in Neurology · doi:10.3389/fneur.2018.00799 · Cited by 20
“Instead, we chose to average the aggregated data, which is a simple, yet effective method to reduce the flaw resulting from pseudoreplication of individual records (147).”
Method
Untitled work
(2015) · HAL · Cited by 20
Visual speech differentially modulates beta, theta, and high gamma bands in auditory cortex
Ganesan, K., Plass, J., Beltz, A. M., Liu, Z., Grabowecky, M., Suzuki, S., Stacey, W. C., Wasade, V. S., Towle, V. L., Tao, J. X., et al. (2021) · European Journal of Neuroscience · doi:10.1111/ejn.15482 · Cited by 21
“This approach has been discussed extensively under the title of ‘ pseudoreplication ’ and can lead to spurious and poorly generalized results (for a discussion, see Aarts et al., 2014; Lazic, 2010; Lazic et al., 2018).”
Metabolic activation of amygdala, lateral septum and accumbens circuits during food anticipatory behavior
Olivo, D., Caba, M., Gonzalez-Lima, F., Rodríguez‐Landa, J. F., & Corona-Morales, A. A. (2016) · Behavioural Brain Research · doi:10.1016/j.bbr.2016.09.015 · Cited by 19
α-Synuclein–induced Kv4 channelopathy in mouse vagal motoneurons drives nonmotor parkinsonian symptoms
Chiu, W., Kovacheva, L., Musgrove, R. E., Arien‐Zakay, H., Koprich, J. B., Brotchie, J. M., Yaka, R., Ben‐Zvi, D., Hanani, M., Roeper, J., et al. (2021) · Science Advances · doi:10.1126/sciadv.abd3994 · Cited by 20
Refinements to rodent head fixation and fluid/food control for neuroscience
Barkus, C., Bergmann, C., Branco, T., Carandini, M., Chadderton, P. T., Galiñanes, G. L., Gilmour, G., Huber, D., Huxter, J. R., Khan, A. G., et al. (2022) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2022.109705 · Cited by 23
“It should also be made clear what the experimental unit ( “ n ” ), of a study is; whilst this is true for all study types, it is of particular importance in longitudinal and electro-physiological studies where the issue of pseudoreplication needs to be avoided (Lazic, 2010).”
Background
Asymmetric dysregulation of glutamate dynamics across the synaptic cleft in a mouse model of Alzheimer’s disease
Brymer, K. J., Hurley, E. P., Barron, J. C., Mukherjee, B., Barnes, J. R., Nafar, F., & Parsons, M. P. (2023) · Acta Neuropathologica Communications · doi:10.1186/s40478-023-01524-x · Cited by 18
“Thus, significant results presented in the present study are unlikely to arise as a result of pseudoreplication [34].”
Result
Brain growth trajectories in mouse strains with central and peripheral serotonin differences: relevance to autism models
Flood, Z. C., Engel, D. L. J., Simon, C. C., Negherbon, K. R., Murphy, L. J., Tamavimok, W., Anderson, G. M., & Janušonis, S. (2012) · Neuroscience · doi:10.1016/j.neuroscience.2012.03.010 · Cited by 17
Dynamic Reconfiguration, Fragmentation, and Integration of Whole-Brain Modular Structure across Depths of Unconsciousness
Standage, D., Areshenkoff, C. N., Nashed, J. Y., Hutchison, R. M., Hutchison, M., Heinke, D. G., Menon, R. S., Everling, S., & Gallivan, J. P. (2020) · Cerebral Cortex · doi:10.1093/cercor/bhaa085 · Cited by 17
Single‐Nucleus RNA‐Sequencing in Brain Tissue
Waag, R., & Bohacek, J. (2023) · Current Protocols · doi:10.1002/cpz1.919 · Cited by 15
“Importantly, different cells are not experimentally independent, and as such do not constitute replicates (Lazic, 2010; Squair et al., 2021).”
Background
Comparison of irrigant activation devices and conventional needle irrigation on smear layer and debris removal in curved canals. (Smear layer removal from irrigant activation using SEM)
Andreani, Y., Gad, B., Cocks, T. C., Harrison, J., Keresztes, M. E., Pomfret, J. K., Rees, E. B., Ma, D., Baloun, B. L., & Rahimi, M. (2021) · Australian Endodontic Journal · doi:10.1111/aej.12482 · Cited by 17
Circadian feeding entrains anticipatory metabolic activity in piriform cortex and olfactory tubercle, but not in suprachiasmatic nucleus
Olivo, D., Caba, M., Gonzalez-Lima, F., Vázquez, A., & Corona-Morales, A. A. (2014) · Brain Research · doi:10.1016/j.brainres.2014.09.054 · Cited by 16
Teaching Experimental Design
Fry, D. (2014) · ILAR Journal · doi:10.1093/ilar/ilu031 · Cited by 17
“…nearly one-half of the ecological papers surveyed were faulty in that experimental entities were taken as independent when they were not (pseudoreplication) and inappropriate statistical analysis was used; more recently, Lazic (2010) has voiced concerns about pseudoreplication in neuroscience.”
BackgroundMethodInfluential
Comment on "Magnetosensitive neurons mediate geomagnetic orientation in Caenorhabditis elegans"
Landler, L., Nimpf, S., Hochstoeger, T., Nordmann, G. C., Papadaki-Anastasopoulou, A., & Keays, D. A. (2018) · eLife · doi:10.7554/elife.30187 · Cited by 16
Potentials and pitfalls of transient in vitro reporter bioassays: interference by vector geometry and cytotoxicity in recombinant zebrafish cell lines
Lungu-Mitea, S., & Lundqvist, J. (2020) · Archives of Toxicology · doi:10.1007/s00204-020-02783-6 · Cited by 15
“Asterisks indicate significance tested in a two-way ANOVA mixed model with Dunnett’s post hoc test (*P 0.05, **P 0.01, ***P 0.001) 1 3 to the pRLx series.”
MethodInfluential
Medaka as a model for seasonal plasticity: Photoperiod-mediated changes in behaviour, cognition, and hormones
Lucon‐Xiccato, T., Montalbano, G., Frigato, E., Loosli, F., Foulkes, N. S., & Bertolucci, C. (2022) · Hormones and Behavior · doi:10.1016/j.yhbeh.2022.105244 · Cited by 15
Evaluation of Published Preclinical Experimental Studies in Medicine: Methodology Issues
Јаnkovic, S., Kapo, B., Šukalo, A., & Mašić, I. (2019) · Medical Archives · doi:10.5455/medarh.2019.73.298-302 · Cited by 15
“Pseudo replication leads to inappropriate testing of hypothesis (because statistical tests for testing difference between groups assume independence of experimental units) and to false precision, as improved estimate after repeating measurements on the same experimental unit just gives more precise results for that ex…”
Background
Methodological issues when using face prototypes: A case study on the Faceaurus dataset
Bovet, J., Tognetti, A., & Pollet, T. V. (2022) · Evolutionary Human Sciences · doi:10.1017/ehs.2022.25 · Cited by 14
“…& Lucas, 2009; Hurlbert, 1984; Kroodsma et al., 2001; McGregor, 2000; Waller et al., 2013), there appears to be a less widespread awareness of this issue in other fields of the behavioural sciences, including evolutionary psychology (but see, Lazic, 2010; Ramírez et al., 2000; Winter, 2011).”
Background
Reporting In Vitro Experiments Responsibly – the RIVER RecommendationsPreprint
The RIVER working group (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/x6aut · Cited by 18
The intrinsic plasticity of medial vestibular nucleus neurons during vestibular compensation—a systematic review and meta-analysis
Wijesinghe, R., & Camp, A. J. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01399-2 · Cited by 14
“This raises the concern of pseudoreplication [45].”
Large-scale organoid study suggests effects of trisomy 21 on early fetal neurodevelopment are more subtle than variability between isogenic lines and experiments
Czerminski, J. T., King, O. D., & Lawrence, J. B. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.972201 · Cited by 16
“3 comparison, to avoid false positives due to pseudoreplication (Lazic, 2010).”
BackgroundMethod
Hemodynamic variability and cerebrovascular control after transient cerebral ischemia
Allan, P. D., Faulkner, J., O’Donnell, T., Lanford, J., Wong, L., Saleem, S., Woolley, B., Lambrick, D., Stoner, L. R., & Tzeng, Y. (2015) · Physiological Reports · doi:10.14814/phy2.12602 · Cited by 13
A Guerilla Guide to Common Problems in 'Neurostatistics': Essential Statistical Topics in Neuroscience.
Smith, P. F. (2017) · PubMed Central · Cited by 14
Improving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2018) · Cancer Reports · doi:10.1002/cnr2.1150 · Cited by 15
“Errors in statistical outputs often arise when the authors (a) do not conduct and report a power calculation(70) or do not distinguish between exploratory and confirmatory analyses;(71) (b) fail to state which statistical tests are used or provide adequate detail about the tests, including the descriptive statistics a…”
Influential
Ecological changes and local knowledge in a giant honey bee (Apis dorsata F.) hunting community in Palawan, Philippines
Matias, D. M., Borgemeister, C., & von Wehrden, H. (2018) · AMBIO · doi:10.1007/s13280-018-1038-7 · Cited by 13
Hierarchical auditory perception for species discrimination and individual recognition in the music frog
Fan, Y., Fang, K., Sun, R., Shen, D., Yang, J., Tang, Y., & Fang, G. (2021) · Current Zoology · doi:10.1093/cz/zoab085 · Cited by 13
“Since pseudoreplication may affect the results of statistical analyses in ecological, animal behavior, and neuroscience studies (Kroodsma et al. 2001; Freeberg and Lucas 2009; Schank and Koehnle 2009; Lazic 2010), we controlled for these possible effects by using multiple stimulus exemplars.”
Background
Evaluation of Preclinical and Clinical Studies Published in Medical Journals of Bosnia and Herzegovina: Methodology Issues
Јаnkovic, S., & Mašić, I. (2020) · Acta Informatica Medica · doi:10.5455/aim.2020.28.4-11 · Cited by 13
“Biological variability between independent experimental units is high, and if the treatments are not randomly assigned, or the experiments are not repeated for sufficient number of times on independent units, individual characteristics on one or a handful of experimental units will be erroneously understood as represe…”
Background
Monitoring the effects of thiamethoxam applied as a seed treatment to winter oilseed rape on the development of bumblebee (Bombus terrestris) colonies
Thompson, H., Coulson, M., Ruddle, N., Wilkins, S., Harrington, P., & Harkin, S. (2015) · Pest Management Science · doi:10.1002/ps.4202 · Cited by 13
Factors Influencing the Temporal and Spatial Patterns of Dogwood Borer (Lepidoptera: Sesiidae) Infestations in Newly Planted Apple Orchards
Frank, D. L., Brewster, C. C., Leskey, T., & Bergh, J. C. (2011) · Environmental Entomology · doi:10.1603/en09309 · Cited by 13
Experimental Design, Inferential Statistics, and Computer Modeling
Tomberlin, J. K., & Benbow, M. E. (2015) · CRC Press eBooks · doi:10.1201/b18156-23 · Cited by 13
Comment on "Stress in Puberty Unmasks Latent Neuropathological Consequences of Prenatal Immune Activation in Mice"
Lazic, S. E. (2013) · Science · doi:10.1126/science.1237793 · Cited by 13
Self-citation
Replications, Comparisons, Sampling and the Problem of Representativeness in Animal Cognition ResearchPreprint
Farrar, B. G., Voudouris, K., & Clayton, N. S. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/2vt4k · Cited by 14
Determining how many cells to average for statistical testing of microscopy experiments
Zweifach, A. (2024) · The Journal of Cell Biology · doi:10.1083/jcb.202401074 · Cited by 12
Cage-induced stereotypic behaviour in laboratory mice covaries with nucleus accumbens FosB/ΔFosB expression
Phillips, D., Choleris, E., Ervin, K. S. J., Fureix, C., Harper, L., Reynolds, K. E., Niel, L., & Mason, G. J. (2015) · Behavioural Brain Research · doi:10.1016/j.bbr.2015.12.035 · Cited by 12
Animal welfare research is fascinating, ethical, and useful—but how can it be more rigorous?
Mason, G. J. (2023) · BMC Biology · doi:10.1186/s12915-023-01793-x · Cited by 13
“Furthermore, other work (e.g. excellent publications by Stanley Lazic, including [4]) identifies pseudoreplication as a common statistical error.”
Background
Systematic heterogenization revisited: Increasing variation in animal experiments to improve reproducibility?
von Kortzfleisch, V. T., & Richter, S. H. (2023) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2023.109992 · Cited by 13
Sound Science: Maintaining Numerical and Statistical Standards in the Pursuit of Noise Exposure Criteria for Marine Mammals
Wright, A. J. (2015) · Frontiers in Marine Science · doi:10.3389/fmars.2015.00099 · Cited by 11
“This is due to the need to avoid using multiple samples that are not independent to generate trends (e.g., Lazic, 2010; Vaux et al., 2012).”
Background
Migration Patterns, Timing, and Seasonal Destinations of Adult Ferruginous Hawks (Buteo regalis)
Watson, J., Banasch, U., Byer, T., Svingen, D. N., Mccready, R., Cruz, M., Hanni, D. J., Lafón‐Terrazas, A., & Gerhardt, R. (2018) · Journal of Raptor Research · doi:10.3356/jrr-17-30.1 · Cited by 11
“Modeling bird ID as a random effect allowed us to address potential pseudoreplication arising from inclusion of 2 yr of data for some birds (Hurlbert 1984, Lazic 2010).”
Statistical Analysis of the Possible Association Between Geomagnetic Storms and Cetacean Mass Strandings
Pulkkinen, A., Moore, K., Zellar, R., Uritskaya, O., Karaköylü, E. M., Uritsky, V. M., & Reeb, D. (2020) · Journal of Geophysical Research Biogeosciences · doi:10.1029/2019jg005441 · Cited by 11
Not all mice are alike: Mixed-strain housing alters social behaviour
Bodden, C., Wewer, M., Kästner, N., Palme, R., Kaiser, S., Sachser, N., & Richter, S. H. (2020) · Physiology & Behavior · doi:10.1016/j.physbeh.2020.113220 · Cited by 10
“Similarly, another study found increased anxiety-like behaviour as well as signs of anhedonia and impaired learning in C57BL/6 mice when housed in high-density mixed-strain groups with DBA/2 mice compared to when raised in same-strain groups housed at lower density [9] (but see [11] for methodological concerns).”
Background
Effects of manipulating slowpoke calcium-dependent potassium channel expression on rhythmic locomotor activity in Drosophila larvae
McKiernan, E. C. (2013) · PeerJ · doi:10.7717/peerj.57 · Cited by 11
“Multiple bursts gathered from the same animal are not independent measures, and dealing with them as such constitutes pseudoreplication [42].”
Background
Neuronal pentraxin 1 depletion delays neurodegeneration and extends life in Sandhoff disease mice
Hooper, A. W. M., Alamilla, J., Venier, R. E., Gillespie, D. C., & Igdoura, S. A. (2016) · Human Molecular Genetics · doi:10.1093/hmg/ddw422 · Cited by 12
“measurements for each individual animal were averaged, and then statistics were performed by animal, rather than by neuron (49,50).”
Method
Crafting statistical analysis plans: A cross‐discipline approach
Cressman, K. A., & Sharp, J. L. (2022) · Stat · doi:10.1002/sta4.528 · Cited by 10
A host of issues: pseudoreplication in host–microbiota studies
Reish, H. M., Dewey, L., & Kirschman, L. J. (2024) · Applied and Environmental Microbiology · doi:10.1128/aem.01033-24 · Cited by 10
“This is similar to the rate of pseudoreplication that Hurlbert found in 1984 (2) but lower than contemporary reviews of primatology and neuroscience (5, 6).”
A call to action to address critical flaws and bias in laboratory animal experiments and preclinical research
Townsend, H. G. G., Osterrieder, K., Jelinski, M. D., Morck, D. W., Waldner, C., Cox, W. R., Gerdts, V., Potter, A. A., Babiuk, L. A., & Cross, J. C. (2025) · Scientific Reports · doi:10.1038/s41598-025-15935-4 · Cited by 15
“In addition, based on our broad sample of the literature, only 21%, 95% CI (14, 28) of comparative laboratory animal experiments even mentioned both randomization and blinding.”
How to avoid the top ten pitfalls in insect conservation and diversity research and minimise your chances of manuscript rejection
Leather, S. R., Basset, Y., & Didham, R. K. (2013) · Insect Conservation and Diversity · doi:10.1111/icad.12066 · Cited by 10
“, 2012), as well as in other areas of the biological sciences (Lazic, 2011; Drummond & Vowler, 2012; Nikinmaa et al., 2012).”
Background
Effects of prenatal stress and monoaminergic perturbations on the expression of serotonin 5-HT4 and adrenergic β2 receptors in the embryonic mouse telencephalon
Chen, A., Kelley, L. D. S., & Janušonis, S. (2012) · Brain Research · doi:10.1016/j.brainres.2012.04.019 · Cited by 11
Wood anatomy of dominant species with contrasting ecological performance in tropical dry forest succession
Romero, E., González, E. J., Meave, J. A., & Terrazas, T. (2019) · Plant Biosystems - An International Journal Dealing with all Aspects of Plant Biology · doi:10.1080/11263504.2019.1651775 · Cited by 10
Tissue plasminogen activator inhibits NMDA-receptor-mediated increases in calcium levels in cultured hippocampal neurons
Robinson, S. D., Lee, T. W., Christie, D. L., & Birch, N. P. (2015) · Frontiers in Cellular Neuroscience · doi:10.3389/fncel.2015.00404 · Cited by 10
“…replicates nested within biological replicates) and were not balanced (not all treatments were in all plates, differing number of technical replicates), the mixed models were fitted by restricted maximum likelihood (REML) instead of general linear model/ANOVA (Littell et al., 2002; Lazic, 2010).”
Method
Single-cell differential expression analysis between conditions within nested settings
Hafner, L., Sturm, G., Lumpp, S., Drton, M., & List, M. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf397 · Cited by 10
“Ignoring this dependence violates the independence assumption that underlies these tests, leading to undere stimated variability and inflated type 1 error, which ultimately compromises the reproducibility of statistical findings [11, 12].”
Grade-control outdoor turning flight of robo-pigeon with quantitative stimulus parameters
Fang, K., Mei, H., Tang, Y., Wang, W., Wang, H., Wang, Z., & Dai, Z. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1143601 · Cited by 9
“Due to the fact that pseudoreplication might affect the conclusions of statistical analyses in ecological, animal behavior and neuroscience studies, the test order of the respective sessions obeyed a pseudorandomized sequence at the present study (Freeberg and Lucas, 2009; Lazic, 2010).”
Method
Olfactory stimulation induces cerebellar vermis activation during sexual learning in male rats
Hernández-Briones, Z. S., García-Bañuelos, P., Hernández‐Aguilar, M. E., López, M.-L., Chacón, A. J. M., Carrillo, P., Coria‐Ávila, G. A., Manzo, J. G., & García, L. I. (2017) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2017.11.003 · Cited by 9
“The pseudoreplication model (nesting factor within brackets) was part of the variation as suggested (García et al., 2015; Herrera-Meza, Manzo, Hernández, Miquel, & García, 2014; Lazic, 2010).”
Method
Multivariate classification of neuroimaging data with nested subclasses: Biased accuracy and implications for hypothesis testing
Jamalabadi, H., Alizadeh, S., Schönauer, M., Leibold, C., & Gais, S. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006486 · Cited by 9
A high-throughput ratiometric method for imaging hypertrophic growth in cultured primary cardiac myocytes
Loonat, A. A., Curtis, M. K., Richards, M. A., Nunez-Alonso, G. A., Michl, J., & Swietach, P. (2019) · Journal of Molecular and Cellular Cardiology · doi:10.1016/j.yjmcc.2019.04.001 · Cited by 9
Development of a toxicokinetic-toxicodynamic model simulating chronic copper toxicity to the Zebra mussel based on subcellular fractionation
Le, T. T. Y., Grabner, D. S., Nachev, M., Garcia, M. R., Balsa‐Canto, E., Peijnenburg, W. J. G. M., Hendriks, A. J., & Sures, B. (2021) · Aquatic Toxicology · doi:10.1016/j.aquatox.2021.106015 · Cited by 9
Activación del cerebelo por estimulación olfativa en ratas macho sexualmente inexpertas
García, L. I., García-Bañuelos, P., Aranda-Abreu, G. E., Meza, G. H., Coria‐Ávila, G. A., & Manzo, J. G. (2014) · Neurología · doi:10.1016/j.nrl.2014.02.002 · Cited by 9
Auditory perception exhibits sexual dimorphism and left telencephalic dominance inXenopus laevis
Fan, Y., Yue, X., Xue, F., Cui, J., Brauth, S. E., Tang, Y., & Fang, G. (2018) · Biology Open · doi:10.1242/bio.035956 · Cited by 10
“…the conclusions of statistical analyses in ecological, animal behavior and neuroscience studies (Freeberg and Lucas, 2009; Kroodsma et al., 2001; Lazic, 2010; Schank and Koehnle, 2009), the possible effects of pseudoreplication were controlled by using multiple stimulus examplars in the present…”
Method
Prostaglandin E2Induces Long-Lasting Inhibition of Noradrenergic Neurons in the Locus Coeruleus and Moderates the Behavioral Response to Stressors
Mukai, Y., Okubo, T. S., Lazarus, M., Ono, D., Tanaka, K. F., & Yamanaka, A. (2023) · Journal of Neuroscience · doi:10.1523/jneurosci.0353-23.2023 · Cited by 9
“To examine the correlation among ROIs collected from the same animal, random intercepts in a linear mixed-effects (LME) model (Lazic, 2010; Aarts et al., 2014; Yu et al., 2022) were incorporated into the statistical analysis of the calcium imaging data.”
Method
The impact of N-back-induced mental workload and time budget on takeover performance
Goodridge, C. M., Gonçalves, R. C., Arabian, A., Horrobin, A., Solernou, A., Lee, Y. T., Bruneau, A., Lee, Y. M., & Merat, N. (2025) · Accident Analysis & Prevention · doi:10.1016/j.aap.2025.108327 · Cited by 8
The Lack of Synapsin Alters Presynaptic Plasticity at Hippocampal Mossy Fibers in Male Mice
Bruentgens, F., Velasquez, L. M., Stumpf, A., Parthier, D., Breustedt, J., Benfenati, F., Milovanović, D., Schmitz, D., & Orlando, M. (2024) · eNeuro · doi:10.1523/eneuro.0330-23.2024 · Cited by 9
“Nested structures introduce correlations in the sample, which have to be accounted for, to avoid in ﬂ ation of false-positive conclusions (Lazic, 2010; Aarts et al., 2014).”
Background
Representations of the intrinsic value of information in mouse orbitofrontal cortex
Bussell, J. J., Badman, R. P., Márton, C. D., Bromberg-Martin, E. S., Abbott, L. F., Rajan, K., & Axel, R. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02377-y · Cited by 13
“We calculated the 95% CI for the decoded accuracy results across the six animals, accounting for dependent measurements across sessions 82 .”
Requirement of ecological replication with independent parallel analysis of each replicate plot to support soil remediation
Sun, Y., Rantalainen, A., Romantschuk, M., & Sinkkonen, A. (2018) · International Biodeterioration & Biodegradation · doi:10.1016/j.ibiod.2018.06.006 · Cited by 8
Statistical experiment design for animal researchPreprint
Sorzano, C. Ó. S. (2023) · doi:10.31219/osf.io/e9s25 · Cited by 9
Maternal gastrointestinal nematode infection enhances spatial memory of uninfected juvenile mouse pups
Noel, S. C., Fortin-Hamel, L., Haque, M., & Scott, M. E. (2022) · Scientific Reports · doi:10.1038/s41598-022-13971-y · Cited by 8
Induction of mandibular tremor using electrolytic lesion of the ventrolateral striatum or using subchronic haloperidol therapy in male rats: An electromyographic comparison
Meza, G. H., Manzo, J. G., Hernández‐Aguilar, M. E., Miquel, M., & García, L. I. (2014) · Neurología (English Edition) · doi:10.1016/j.nrleng.2013.10.019 · Cited by 7
Relationships among variables and their equilibrium values: caveats of time‐less interpretation
Janušonis, S. (2011) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/j.1469-185x.2011.00194.x · Cited by 7
“…of statistical analysis is undoubtedly dismal in some (otherwise highly sophisticated) biology fields (Nakagawa & Cuthill, 2007; Janušonis, 2009; Lazic, 2008, 2010; Nakagawa & Hauber, 2011), the problem discussed in this review is different and will remain unsolved even when statistical…”
Background
Susceptibility allele-specific loss of miR-1324-mediated silencing of the INO80B chromatin-assembly complex gene in pre-eclampsia
Oudejans, C. B. M., Michel, O. J., Janssen, R., Habets, R., Poutsma, A., Sistermans, E. A., Weiss, M. M., Incarnato, D., Oliviero, S., Kleiverda, G., et al. (2014) · Human Molecular Genetics · doi:10.1093/hmg/ddu423 · Cited by 7
“To prevent the problem of pseudoreplication (29), independent measurements separated in time are usually considered essential for hierarchical systems (such as cell cultures), i.”
Background
First-Year Migration and Natal Region Fidelity of Immature Ferruginous Hawks
Watson, J., Banasch, U., Byer, T., Svingen, D. N., Mccready, R., Hanni, D. J., & Gerhardt, R. (2019) · Journal of Raptor Research · doi:10.3356/jrr-18-32 · Cited by 7
“Different patterns and areas of use may affect age-specific survival and thus be important to conservation needs of immature hawks (Mañosa et al. 1998, McIntyre et al. 2006, Cadahia et al. 2010, Hunt et al. 2017).”
Background
Activation of the cerebellum by olfactory stimulation in sexually naive male rats
García, L. I., García-Bañuelos, P., Aranda-Abreu, G. E., Meza, G. H., Coria‐Ávila, G. A., & Manzo, J. G. (2015) · Neurología (English Edition) · doi:10.1016/j.nrleng.2014.02.004 · Cited by 7
“analysis.(17) A Tukey post hoc test was completed to determine any inter-group differences in lobules.”
Method
Using meta-analysis for evidence synthesis: The case of incomplete neutralization in GermanPreprint
Nicenboim, B., Roettger, T. B., & Vasishth, S. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/p5a4z · Cited by 9
Pseudoreplication in Primate Communication Research: 10 Years On
Whitehouse, J., Clark, P. R., Micheletta, J., Liebal, K., Slocombe, K. E., & Waller, B. M. (2023) · International Journal of Primatology · doi:10.1007/s10764-023-00399-y · Cited by 7
“Since then, a variety of scientific fields have highlighted pseudorep-lication as an ongoing statistical issue (Freeberg & Lucas, 2009; Heffner et al ., 1996; Johnson & Freeberg, 2016; Kroodsma et al ., 2001; Lazic, 2010; Waller et al ., 2013).”
Altered APP trafficking drives amyloidogenic processing in primary neurons from the AppNL-F knock-in mouse model of Alzheimer's disease
Yu, Y., Zhou, R. Z., Nilsson, P., Winblad, B. G., Tjernberg, L. O., & Schedin‐Weiss, S. (2025) · Neurobiology of Disease · doi:10.1016/j.nbd.2025.107129 · Cited by 5
Yawning and Its Temperature-Dependent Modulation in Leopard Geckos
Kotake, K. T., Yamaguchi, S. T., Mukai, Y., Zhou, Z., & Norimoto, H. (2024) · ZOOLOGICAL SCIENCE · doi:10.2108/zs230123 · Cited by 6
“2A), a generalized linear mixed-effects model (GLMM) was used to reduce the impact of individual differences on the statistics (Lazic, 2010; Yu et al., 2022; Mukai et al., 2023).”
Studying Cellular Signal Transduction with OMIC Technologies
Landry, B. D., Clarke, D. C., & Lee, M. J. (2015) · Journal of Molecular Biology · doi:10.1016/j.jmb.2015.07.021 · Cited by 8
“The use of inappropriate replication strategies is called “pseudoreplication” and is a common problem in many reported studies [89].”
Automated pulse discrimination of two freely-swimming weakly electric fish and analysis of their electrical behavior during dominance contest
Guariento, R. T., Mosqueiro, T. S., Matias, P., Cesarino, V. B., Almeida, L. O. B., Slaets, J. F. W., Maia, L. P., & Pinto, R. D. (2016) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2017.02.001 · Cited by 6
“Because the distribution of IPIs in most cases resemble a bell-shaped distribution (see Figure S6) and because failing the hypothesis of normally distributed samples induces less errors than the existence of correlation among samples (Lazic, 2010), we used a”
Changes in multiunit activity pattern in cerebellar cortex associated to olfactory cues during sexual learning in male rats
Rodríguez, A. T., Vásquez-Celaya, L., Coria‐Ávila, G. A., Pérez-Estudillo, C. A., Aranda-Abreu, G. E., Carrillo, P., Manzo, J. G., & García, L. I. (2018) · Neuroscience Letters · doi:10.1016/j.neulet.2018.09.063 · Cited by 6
The long-term impact of infant rearing background on the behavioural and physiological stress response of adult common marmosets (Callithrix jacchus)
Ash, H., Smith, T. E., & Buchanan‐Smith, H. M. (2020) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2020.105169 · Cited by 6
Enhancing rigor and reproducibility in maternal immune activation models: practical considerations and predicting resilience and susceptibility using baseline immune responsiveness before pregnancyPreprint
Estes, M. L., Farrelly, K., Cameron, S., Aboubechara, J. P., Haapanen, L. A. D., Schauer, J. D., Horta, A., Prendergast, K., MacMahon, J. A., Shaffer, C. I., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/699983 · Cited by 4
“grooming in MIA offspring using 30mg/kg pure poly(I:C), the difference was not statistically significant due to the high amount of variance (Figure 1i), which was also evident when we accounted for litter effects (Figure 1j) (26).”
Method
Response of tomato growth and transpiration under adequate elevated CO2 concentrations coupled with irrigation regimes
Akhlaq, M., Miao, H., Zhang, C., Xue, R., Yan, H., Hameed, M. U., Li, X., Ren, J., Chauhdary, J. N., & Rehman, M. M. U. (2025) · Irrigation Science · doi:10.1007/s00271-025-01034-9 · Cited by 5
“(Hurlbert 1984; Lazic 2010 ).”
Linear mixed-effects models for the analysis of high-density electromyography with application to diabetic peripheral neuropathy
Andreis, F. R., Favretto, M. A., Cossul, S., Nakamura, L. R., Barbetta, P. A., & Marques, J. L. B. (2020) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-020-02181-1 · Cited by 5
Aortic valve leaflet assessment to inform novel bioinspired materials: Understanding the impact of collagen fibres on the tissue's mechanical behaviour
Hughes, C., Whelan, A., O'Reilly, D., Campbell, E. M., & Lally, C. (2024) · Journal of the mechanical behavior of biomedical materials/Journal of mechanical behavior of biomedical materials · doi:10.1016/j.jmbbm.2024.106881 · Cited by 5
Synaptic alterations associated with disrupted sensory encoding in a mouse model of tauopathy
Meftah, S., Cavallini, A., Murray, T. K., Jankowski, Ł., Bose, S., Ashby, M. C., Brown, J. T., & Witton, J. (2024) · Brain Communications · doi:10.1093/braincomms/fcae134 · Cited by 6

Authors: Lazic, S. E.
Affiliations: F. Hoffmann-La Roche AG (CH)
Corresponding author: Lazic, Stanley E
ORCID: Lazic, Stanley E
Venue: BMC Neuroscience
Publisher: Springer Science and Business Media LLC
Published: 2010-01-14
Volume: 11
Issue: 1
Pages: 5-5
ISSN: 1471-2202 (ISSN-L)
DOI: 10.1186/1471-2202-11-5
PMC: PMC2817684
OpenAlex: W2109626712
Cited by: 381
References: 90
Citations by year: 2026: 68 · 2025: 30 · 2024: 19 · 2023: 32 · 2022: 20 · 2021: 21 · 2020: 31 · 2019: 16 · 2018: 23 · 2017: 22 · 2016: 15 · 2015: 19 · 2014: 20 · 2013: 17 · 2012: 19
FWCI: 12.28
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
creativecommons.org/licenses/by/2.0 (tdm)
Locations: BMC Neuroscience (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
KAIST Institutional Repository (KAIST) (submittedVersion)
King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) (submittedVersion, other-oa)
Funders: Cancer Research UK (10218)
University of Oxford
Cambridge Commonwealth Trust
King Abdullah University of Science and Technology (KUK‐C1‐013‐04)
Korea Science and Engineering Foundation
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: pseudoreplication, statistical dependence, nested data, hierarchical data, repeated measures, time series correlation, spatial correlation, statistical analysis errors, sample size reporting, degrees of freedom, p-values
MeSH terms: Analysis of Variance, Animals, Computer Simulation, Neurosciences, Periodicals as Topic, Time Factors, Models, Statistical, Linear Models, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?

Abstract

Overview

Integrity signals

Figures

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks