A protocol for data exploration to avoid common statistical problems: Data exploration
Open accessAbstract
1. While teaching statistics to ecologists, the lead authors of this paper have noticed common statistical problems. If a random sample of their work (including scientific papers) produced before doing these courses were selected, half would probably contain violations of the underlying assumptions of the statistical techniques employed. 2. Some violations have little impact on the results or ecological conclusions; yet others increase type I or type II errors, potentially resulting in wrong ecological conclusions. Most of these violations can be avoided by applying better data exploration. These problems are especially troublesome in applied ecology, where management and policy decisions are often at stake. 3. Here, we provide a protocol for data exploration; discuss current tools to detect outliers, heterogeneity of variance, collinearity, dependence of observations, problems with interactions, double zeros in multivariate analysis, zero inflation in generalized linear modelling, and the correct type of relationships between dependent and independent variables; and provide advice on how to address these problems when they arise. We also address misconceptions about normality, and provide advice on data transformations. 4. Data exploration avoids type I and type II errors, among other problems, thereby reducing the chance of making wrong ecological conclusions and poor recommendations. It is therefore essential for good quality management and policy based on statistical analyses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Collinearity among the moderators was inspected with variance inflation factor and found not to be an issue (VIF < 1.7 for all moderators; threshold = 3).
“Collinearity among our moderators was inspected with variance inflation factor (VIF; Zuur, Ieno & Elphick, 2010) and found not to be an issue in our model (Table S6; VIF <1.7 for all moderators – threshold value = 3; Zuur et al., 2010).”
From Acoustic indices as proxies for biodiversity: a meta‐analysis (2022) · cites this paper as Zuur, Ieno & Elphick, 2010
Collinearity among the moderators was inspected with variance inflation factor and found not to be an issue (VIF < 1.7 for all moderators; threshold = 3).
“Collinearity among our moderators was inspected with variance inflation factor (VIF; Zuur, Ieno & Elphick, 2010) and found not to be an issue in our model (Table S6; VIF <1.7 for all moderators – threshold value = 3; Zuur et al., 2010).”
From Acoustic indices as proxies for biodiversity: a meta‐analysis (2022) · cites this paper as Zuur et al., 2010
Collinearity between independent variables was assessed prior to analysis using variance inflation factors (VIF) and Spearman rank correlation tests.
“Collinearity between independent variables was assessed prior to analysis using variance inflation factors (VIF) and Spearman rank correlation tests [54].”
From Effects of Vegetation Structure on the Location of Lion Kill Sites in African Thicket (2016) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495 · see citing works
Draper, N. R., & Smith, H. (2005) · Technometrics · doi:10.1198/tech.2005.s303 · Cited by 17,993 · see citing works
Mixed effects models and extensions in ecology with R
Zuur, A. F., Ieno, E. N., Walker, N., Saveliev, A. A., & Smith, G. M. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-87458-6 · Cited by 17,963 · see citing works
Self-citationJolliffe, I. T. (2005) · Wiley eBooks · doi:10.1002/0470013192.bsa501 · Cited by 14,522 · see citing works
Generalized Additive Models: An Introduction with R
Wood, S. N. (2017) · doi:10.1201/9781315370279 · Cited by 13,852 · see citing works
Mixed-Effects Models in Sand S-PLUS
Pinheiro, J. C., & Bates, D. M. (2000) · Statistics and Computing · doi:10.1007/978-1-4419-0318-1 · Cited by 11,187 · see citing works
Experimental Design and Data Analysis for Biologists
Quinn, G. P., & Keough, M. J. (2002) · doi:10.1017/cbo9780511806384 · Cited by 10,674 · see citing works
Mixed-Effects Models in S and S-PLUS
(2001) · Technometrics · doi:10.1198/tech.2001.s574 · Cited by 9,278 · see citing works
Pseudoreplication and the Design of Ecological Field Experiments
Hurlbert, S. H. (1984) · Ecological Monographs · doi:10.2307/1942661 · Cited by 8,377 · see citing works
Chiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced textbooks in control and signal processing · doi:10.1007/978-1-4471-0347-9_4 · Cited by 6,684 · see citing works
Regression Analysis of Count Data
Cameron, A. C., & Trivedi, P. K. (2013) · doi:10.1017/cbo9781139013567 · Cited by 6,640 · see citing works
Regression Analysis of Count Data: Author Index
Cameron, A., & Trivedi, P. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9780511814365 · Cited by 6,406 · see citing works
Introduction to Linear Regression Analysis.
Assunção, R., Sampson, P. D., Montgomery, D. C., & Peck, E. A. (1993) · Journal of the American Statistical Association · doi:10.2307/2290746 · Cited by 5,389 · see citing works
Draper, N. R., & Smith, H. (1998) · Wiley series in probability and statistics · doi:10.1002/9781118625590 · Cited by 5,086 · see citing works
Wood, S. N. (2006) · doi:10.1201/9781420010404 · Cited by 4,953 · see citing works
Forecasting, Structural Time Series Models and the Kalman Filter
Fildes, R. A., Harvey, A. C., West, M., & Harrison, J. (1991) · Journal of the Operational Research Society · doi:10.2307/2583225 · Cited by 4,683 · see citing works
Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781119513469 · Cited by 4,639 · see citing works
Forecasting, Structural Time Series Models and the Kalman Filter
Harvey, A. C. (1990) · doi:10.1017/cbo9781107049994 · Cited by 4,195 · see citing works
Introduction to Time Series and Forecasting
Brockwell, P. J., & Davis, R. A. (2002) · Springer texts in statistics · doi:10.1007/b97391 · Cited by 3,914 · see citing works
Zuur, A. F., Ieno, E. N., & Smith, G. M. (2007) · Statistics in the health sciences · doi:10.1007/978-0-387-45972-1 · Cited by 2,460 · see citing works
Self-citationMeasurement Error in Nonlinear Models
Carroll, R. J., Ruppert, D., Stefanski, L. A., & Crainiceanu, C. M. (2006) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420010138 · Cited by 2,125 · see citing works
Applied Regression Analysis and Generalized Linear Models
Fox, J. (2008) · Cited by 2,051 · see citing works
Hilbe, J. M. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511811852.007 · Cited by 1,998 · see citing works
Canonical correspondence analysis and related multivariate methods in aquatic ecology
ter Braak, C. J. F., & Verdonschot, P. F. M. (1995) · Aquatic Sciences · doi:10.1007/bf00877430 · Cited by 1,944 · see citing works
Sarkar, D. (2008) · Springer eBooks · doi:10.1007/978-0-387-75969-2 · Cited by 1,792 · see citing works
Applied Mixed Models in Medicine
Brown, H., & Presco, R. (2000) · Technometrics · doi:10.1080/00401706.2000.10485742 · Cited by 1,699 · see citing works
Cleveland, W. S. (1993) · Cited by 1,599 · see citing works
(1977) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,548 · see citing works
(2004) · Choice Reviews Online · doi:10.5860/choice.42-2265 · Cited by 1,385 · see citing works
Measurement error in nonlinear models: a modern perspective
Carroll, R. J. (2006) · Cited by 1,374 · see citing works
Ellison, A. M. (2004) · Ecology Letters · doi:10.1111/j.1461-0248.2004.00603.x · Cited by 791 · see citing works
Hilbe, J. M. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511811852 · Cited by 756 · see citing works
Contemporary Statistical Models for the Plant and Soil Sciences
Schabenberger, O., & Pierce, F. J. (2001) · CRC Press eBooks · doi:10.1201/9781420040197 · Cited by 717 · see citing works
Contemporary statistical models for the plant and soil sciences
Schabenberger, O., & Pierce, F. J. (2002) · Choice Reviews Online · doi:10.5860/choice.40-0345 · Cited by 683 · see citing works
Applied Mixed Models in Medicine
Brown, H., & Prescott, R. J. (2006) · Wiley eBooks · doi:10.1002/0470023589 · Cited by 567 · see citing works
Time-Series Analysis: Regression Techniques.
Pampel, F. C., & Ostrom, C. W. (1980) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2065616 · Cited by 499 · see citing works
Ecological information from spatial patterns of plants: insights from point process theory
Law, R., Illian, J., Burslem, D. F. R. P., et al. (2009) · Journal of Ecology · doi:10.1111/j.1365-2745.2009.01510.x · Cited by 476 · see citing works
Information theory and hypothesis testing: a call for pluralism
Stephens, P. A., Buskirk, S. W., Hayward, G. D., & del Rio, C. M. (2005) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2005.01002.x · Cited by 361 · see citing works
Measurement Error in Nonlinear Models: A Modern Perspective, Second Edition
Carroll, R. J., Ruppert, D., Stefanski, L. A., et al. (2006) · Cited by 334 · see citing works
Doing more good than harm – Building an evidence-base for conservation and environmental management
Pullin, A. S., & Knight, T. (2009) · Biological Conservation · doi:10.1016/j.biocon.2009.01.010 · Cited by 295 · see citing works
Winter management of Californian rice fields for waterbirds
Elphick, C. S., & Oring, L. W. (1998) · Journal of Applied Ecology · doi:10.1046/j.1365-2664.1998.00274.x · Cited by 260 · see citing works
Self-citationGelman, A., Pasarica, C., & Dodhia, R. M. (2002) · The American Statistician · doi:10.1198/000313002317572790 · Cited by 223 · see citing works
Ostrom, C. W. (1990) · Sage eBooks · doi:10.4135/9781412986366 · Cited by 197 · see citing works
Hall, D. B. (2008) · Quarterly Publications of the American Statistical Association · Cited by 196 · see citing works
Zuur, A. F., Ieno, E. N., & Meesters, E. H. (2009) · Springer eBooks · doi:10.1007/978-0-387-93837-0 · Cited by 176 · see citing works
Self-citationAPPLIED MIXED MODELS IN MEDICINE, 2ND ED.
Homan, S. M. (2007) · Shock · doi:10.1097/01.shk.0000278487.47934.45 · Cited by 176 · see citing works
Conservation implications of flooding rice fields on winter waterbird communities
Elphick, C. S., & Oring, L. W. (2002) · Agriculture Ecosystems & Environment · doi:10.1016/s0167-8809(02)00022-1 · Cited by 154 · see citing works
Self-citationUsing SiZer to detect thresholds in ecological data
Sonderegger, D. L., Wang, H., Clements, W. H., & Noon, B. R. (2008) · Frontiers in Ecology and the Environment · doi:10.1890/070179 · Cited by 142 · see citing works
Problem Solving: A Statistician's Guide.
Kafandaris, S. P., & Chatfield, C. (1996) · Journal of the Operational Research Society · doi:10.2307/3010293 · Cited by 141 · see citing works
Roberts, P. D., Stewart, G. B., & Pullin, A. S. (2006) · Biological Conservation · doi:10.1016/j.biocon.2006.04.034 · Cited by 137 · see citing works
Koper, N., & Manseau, M. (2009) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2009.01642.x · Cited by 133 · see citing works
Gjerdrum, C., Elphick, C. S., & Rubega, M. A. (2005) · Ornithological Applications · doi:10.1650/7723.1 · Cited by 122 · see citing works
Self-citationGjerdrum, C., Elphick, C. S., & Rubega, M. A. (2005) · Ornithological Applications · doi:10.1093/condor/107.4.849 · Cited by 71 · see citing works
Self-citationStatistics: Reasoning on Uncertainty, and the Insignificance of Testing Null
Läärä, E. (2009) · Annales Zoologici Fennici · doi:10.5735/086.046.0206 · Cited by 44 · see citing works
Gjerdrum, C., Elphick, C. S., & Rubega, M. A. (2008) · The Auk · doi:10.1525/auk.2008.07029 · Cited by 27 · see citing works
Self-citationRemarks on the taking and recording of biometric measurements in bird ringing
Morgan, J. H. (2004) · Ring · doi:10.2478/v10050-008-0058-2 · Cited by 14 · see citing works
Problem Solving: A Statistician's Guide
Donnelly, P. P., & Chatfield, C. (1990) · Technometrics · doi:10.2307/1269123 · Cited by 5 · see citing works
Correcting for spatial autocorrelation in sequential sampling
Robinson, A. P., & Hamann, J. D. (2008) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2008.01485.x · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Legendre, P. & Legendre, L. (1998) Numerical Ecology. Second English Edition. Elsevier, Amsterdam.
- Sarkar, D. (2008) Lattice: Multivariate Data Visualization with R. Springer, New York.
Cited by
Sorted by citations
A brief introduction to mixed effects modelling and multi-model inference in ecology
Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018) · PeerJ · doi:10.7717/peerj.4794 · Cited by 2,094 · see citing works
Robustness of linear mixed-effects models to violations of distributional assumptions
Schielzeth, H., Dingemanse, N. J., Nakagawa, S., Westneat, D. F., Allegue, H., Teplitsky, C., Réale, D., Dochtermann, N. A., Garamszegi, L. Z., & Araya‐Ajoy, Y. G. (2020) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13434 · Cited by 1,393 · see citing works
Violating the normality assumption may be the lesser of two evils
Knief, U., & Forstmeier, W. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01587-5 · Cited by 654 · see citing works
Contact with blue-green spaces during the COVID-19 pandemic lockdown beneficial for mental health.
Pouso, S., Borja, Á., Fleming, L. E., Gómez-Baggethun, E., White, M. P., & Uyarra, M. C. (2021) · Science of The Total Environment · doi:10.1016/j.scitotenv.2020.143984 · Cited by 604 · see citing works
Applications of machine learning in animal behaviour studies
Valletta, J. J., Torney, C., Kings, M., Thornton, A., & Madden, J. (2017) · Animal Behaviour · doi:10.1016/j.anbehav.2016.12.005 · Cited by 505 · see citing works
Nested by design: model fitting and interpretation in a mixed model era
Schielzeth, H., & Nakagawa, S. (2013) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2012.00251.x · Cited by 394 · see citing works
Compa, M., Ventero, A., Iglesias, M., & Deudero, S. (2018) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2018.01.009 · Cited by 278 · see citing works
Acoustic indices as proxies for biodiversity: a meta‐analysis
Alcocer, I., Lima, H., Sugai, L. S. M., & Llusia, D. (2022) · Biological Reviews · doi:10.1111/brv.12890 · Cited by 273 · see citing works
“Collinearity among our moderators was inspected with variance inflation factor (VIF; Zuur, Ieno & Elphick, 2010) and found not to be an issue in our model (Table S6; VIF <1.7 for all moderators – threshold value = 3; Zuur et al., 2010).”
MethodInfluentialThe global distribution of known and undiscovered ant biodiversity
Kass, J. M., Guénard, B., Dudley, K. L., Jenkins, C. N., Azuma, F., Fisher, B. L., Parr, C. L., Gibb, H., Longino, J. T., Ward, P. S., et al. (2022) · Science Advances · doi:10.1126/sciadv.abp9908 · Cited by 194 · see citing works
Effects of Vegetation Structure on the Location of Lion Kill Sites in African Thicket
Davies, A. B., Tambling, C. J., Kerley, G. I. H., & Asner, G. P. (2016) · PLOS ONE · doi:10.1371/journal.pone.0149098 · Cited by 97 · see citing works
Pennino, M. G., Bachiller, E., Lloret-Lloret, E., Albo-Puigserver, M., Esteban, A., Jadaud, A., Bellido, J. M., & Coll, M. (2020) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2020.111399 · Cited by 95 · see citing works
Boops boops as a Bioindicator of Microplastic Pollution along the Spanish Catalan Coast
Garcia-Garin, O., Vighi, M., Aguilar, A., Tsangaris, C., Digka, N., Kaberi, H., & Borrell, A. (2019) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2019.110648 · Cited by 78 · see citing works
Rapidly obtained ecosystem indicators from coral reef soundscapes
Freeman, L., & Freeman, S. (2016) · Marine Ecology Progress Series · doi:10.3354/meps11938 · Cited by 66 · see citing works
(2009) · doi:10.1787/20725302 · Cited by 62 · see citing works
Ou, Y., Belzer, C., Smidt, H., & de Weerth, C. (2022) · Gut Microbes · doi:10.1080/19490976.2022.2038853 · Cited by 54 · see citing works
Martinsen, K. T., Andersen, M. R., & Sand-Jensen, K. (2019) · Hydrobiologia · doi:10.1007/s10750-018-3737-2 · Cited by 45 · see citing works
Husbandry and enclosure influences on penguin behavior and conservation breeding
Marshall, A. R., Deere, N. J., Little, H. A., Snipp, R., Goulder, J., & Mayer-Clarke, S. (2016) · Zoo Biology · doi:10.1002/zoo.21313 · Cited by 33 · see citing works
Widespread ecological responses and cascading effects of the 2021 western North American heatwave
Baum, J. K., Slein, M. A., Garen, J. C., Sang, Z., Emry, S., Goodwin, K. J. A., Collins, C. G., Lewthwaite, J., Tseng, M., Michaletz, S. T., et al. (2026) · Nature Ecology & Evolution · doi:10.1038/s41559-026-02987-6 · Cited by 15 · see citing works
Spatial ecology of the stone marten in an Alpine area: combining camera-trapping and genetic surveys
Balestrieri, A., Mosini, A., Fonda, F., Piana, M., Tirozzi, P., Ruiz-González, A., Capelli, E., Vergara, M., Chueca, L. J., Chiatante, G., et al. (2021) · Mammal Research · doi:10.1007/s13364-021-00564-9 · Cited by 11 · see citing works
Assessing data bias in visual surveys from a cetacean monitoring programme
Oliveira-Rodrigues, C., Correia, A. M., Valente, R., Gil, Á., Gandra, M., Liberal, M., Rosso, M., Pierce, G., & Sousa-Pinto, I. (2022) · Scientific Data · doi:10.1038/s41597-022-01803-7 · Cited by 10 · see citing works
Dorph, A., & McDonald, P. G. (2017) · PLOS ONE · doi:10.1371/journal.pone.0179337 · Cited by 7 · see citing works
Norris, D., Sabadini Carvalho, T., Guerrero, A. M., Sobral Escada, M. I., Alencar, A., Kimbrough, L., & Butler, R. A. (2022) · Tropical Conservation Science · doi:10.1177/19400829221132193 · Cited by 6 · see citing works
- Authors
- Zuur, A. F., Ieno, E. N., & Elphick, C. S.
- Affiliations
- University of Aberdeen (GB)
- University of Connecticut (US)
- Corresponding author
- Zuur, Alain F.
- Venue
- Methods in Ecology and Evolution
- Publisher
- Wiley
- Published
- 2009-11-13
- Volume
- 1
- Issue
- 1
- Pages
- 3-14
- ISSN
- 2041-210X (ISSN-L)
- OpenAlex
- W2111684210
- Cited by
- 8,228
- References
- 59
- Citations by year
- 2026: 653 · 2025: 710 · 2024: 737 · 2023: 721 · 2022: 766 · 2021: 833 · 2020: 788 · 2019: 679 · 2018: 609 · 2017: 489 · 2016: 376 · 2015: 283 · 2014: 228 · 2013: 147 · 2012: 116
- FWCI
- 56.44
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Methods in Ecology and Evolution (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- exploratory data analysis, statistical assumptions, type I and type II errors, outliers, heteroscedasticity, multicollinearity, autocorrelation, interaction effects, double zeros, zero-inflated data, generalized linear models, normality, data transformation, applied ecology, statistical misconceptions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
