The DOI matches its Crossref record; 84 of the 85 references it cites were found in a registry or an index.
Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists.
Abstract
Repeatability (more precisely the common measure of repeatability, the intra-class correlation coefficient, ICC) is an important index for quantifying the accuracy of measurements and the constancy of phenotypes. It is the proportion of phenotypic variation that can be attributed to between-subject (or between-group) variation. As a consequence, the non-repeatable fraction of phenotypic variation is the sum of measurement error and phenotypic flexibility. There are several ways to estimate repeatability for Gaussian data, but there are no formal agreements on how repeatability should be calculated for non-Gaussian data (e.g. binary, proportion and count data). In addition to point estimates, appropriate uncertainty estimates (standard errors and confidence intervals) and statistical significance for repeatability estimates are required regardless of the types of data. We review the methods for calculating repeatability and the associated statistics for Gaussian and non-Gaussian data. For Gaussian data, we present three common approaches for estimating repeatability: correlation-based, analysis of variance (ANOVA)-based and linear mixed-effects model (LMM)-based methods, while for non-Gaussian data, we focus on generalised linear mixed-effects models (GLMM) that allow the estimation of repeatability on the original and on the underlying latent scale. We also address a number of methods for calculating standard errors, confidence intervals and statistical significance; the most accurate and recommended methods are parametric bootstrapping, randomisation tests and Bayesian approaches. We advocate the use of LMM- and GLMM-based approaches mainly because of the ease with which confounding variables can be controlled for. Furthermore, we compare two types of repeatability (ordinary repeatability and extrapolated repeatability) in relation to narrow-sense heritability. This review serves as a collection of guidelines and recommendations for biologists to calculate repeatability and heritability from both Gaussian and non-Gaussian data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The survey was based on the validated International Hospital Outcomes Study questionnaire. 25”
From Nurses’ Shift Length and Overtime Working in 12 European Countries: The Association With Perceived Quality of Care and Patient Safety (2014) · cites this paper as [25]
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Modern Applied Statistics with S
Venables, W. N., & Ripley, B. D. (2002) · Statisctics and computing/Statistics and computing · doi:10.1007/978-0-387-21706-2 · Cited by 25,941
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Introduction to Quantitative Genetics.
Edwards, A. W. F., & Falconer, D. S. (1982) · Biometrics · doi:10.2307/2529912 · Cited by 21,447
Biometry: The Principles and Practice of Statistics in Biological Research
Sokal, R. R., & Rohlf, F. J. (1969) · Cited by 20,963
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
Generalized linear mixed models: a practical guide for ecology and evolution
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2008.10.008 · Cited by 8,910
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903
Genetics and Analysis of Quantitative Traits
(2002) · Journal of Animal Breeding and Genetics · doi:10.1046/j.1439-0388.2002.00356.x · Cited by 6,176
MCMC Methods for Multi-Response Generalized Linear Mixed Models: The MCMCglmm R Package
Hadfield, J. D. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i02 · Cited by 4,900
Randomization, Bootstrap and Monte Carlo Methods in Biology
Manly, B. F. J. (2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315273075 · Cited by 4,209
Effect size, confidence interval and statistical significance: a practical guide for biologists
Nakagawa, S., & Cuthill, I. C. (2007) · Biological Reviews · doi:10.1111/j.1469-185x.2007.00027.x · Cited by 3,835
Integrating animal temperament within ecology and evolution
Réale, D. J. F., Reader, S. M., Sol, D., McDougall, P. T., & Dingemanse, N. J. (2007) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/j.1469-185x.2007.00010.x · Cited by 3,587
Linear Mixed Models for Longitudinal Data
Verbeke, G. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-2294-1_3 · Cited by 3,506
Behavioral syndromes: an ecological and evolutionary overview
Sih, A., Bell, A. M., & Johnson, J. C. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2004.04.009 · Cited by 3,327
Generalized, Linear, and Mixed Models
McCulloch, C. E., & Searle, S. R. (2000) · Wiley series in probability and statistics · doi:10.1002/0471722073 · Cited by 3,057
Simple means to improve the interpretability of regression coefficients
Schielzeth, H. (2010) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2010.00012.x · Cited by 3,027 · cited 1×
“The single most representative repeatability is the repeatability at the mean of the covariate (Schielzeth, 2010).”
BackgroundSelf-citationLittell, R. C., Milliken, G. A., Stroup, W. W., Wolfinger, R. D., & Schabenberger, O. (2006) · CERN Document Server (European Organization for Nuclear Research) · Cited by 2,924
Unrepeatable Repeatabilities: A Common Mistake
Lessells, C. M., & Boag, P. T. (1987) · The Auk · doi:10.2307/4087240 · Cited by 2,919
Linear Mixed Models for Longitudinal Data
Molenberghs, G., & Verbeke, G. N. (2000) · Springer series in statistics · doi:10.1007/978-1-4419-0300-6 · Cited by 2,681
The repeatability of behaviour: a meta-analysis
Bell, A. M., Hankison, S. J., & Laskowski, K. L. (2009) · Animal Behaviour · doi:10.1016/j.anbehav.2008.12.022 · Cited by 2,146
Behavioral Syndromes: An Integrative Overview
Sih, A., Bell, A. M., Johnson, J. C., & Ziemba, R. E. (2004) · The Quarterly Review of Biology · doi:10.1086/422893 · Cited by 2,108
Heritability in the genomics era — concepts and misconceptions
Visscher, P. M., Hill, W. G., & Wray, N. R. (2008) · Nature Reviews Genetics · doi:10.1038/nrg2322 · Cited by 1,921
Pigliucci, M. (2001) · Johns Hopkins University Press eBooks · doi:10.56021/9780801867880 · Cited by 1,833
Proceedings of the Royal Society B: Biological Sciences
(2013) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb · Cited by 1,593
Behavioural reaction norms: animal personality meets individual plasticity
Dingemanse, N. J., Kazem, A. J. N., Réale, D. J. F., & WRIGHT, J. M. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2009.07.013 · Cited by 1,506
Immunocompetence, ornamentation, and viability of male barn swallows ( Hirundo rustica )
Saino, N., Bolzern, A. M., & Møller, A. P. (1997) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.94.2.549 · Cited by 1,357
Phenotypic Plasticity: Beyond Nature and Nurture
Pigliucci, M. (2001) · Medical Entomology and Zoology · Cited by 1,225
Introduction to Quantitative Genetics
P.B., S. (1963) · Poultry Science · doi:10.3382/ps.0420547 · Cited by 1,053
Randomization, Bootstrap and Monte Carlo Methods in Biology
Robinson, A. P. (2007) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2007.00485_5.x · Cited by 1,020
The development of animal personality: relevance, concepts and perspectives
Stamps, J. A., & Groothuis, T. G. G. (2009) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/j.1469-185x.2009.00103.x · Cited by 979
Conclusions beyond support: overconfident estimates in mixed models
Schielzeth, H., & Forstmeier, W. (2009) · Behavioral Ecology · doi:10.1093/beheco/arn145 · Cited by 952
A simple method for distinguishing within- versus between-subject effects using mixed models
van de Pol, M., & Wright, J. (2009) · Animal Behaviour · doi:10.1016/j.anbehav.2008.11.006 · Cited by 951
The evolutionary ecology of individual phenotypic plasticity in wild populations
Nussey, D. H., Wilson, A. J., & Brommer, J. E. (2007) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2007.01300.x · Cited by 929
Why environmental scientists are becoming Bayesians
Clark, J. S. (2004) · Ecology Letters · doi:10.1111/j.1461-0248.2004.00702.x · Cited by 887
Partitioning Variation in Multilevel Models
Goldstein, H., Browne, W., & Rasbash, J. (2002) · Understanding Statistics · doi:10.1207/s15328031us0104_02 · Cited by 874 · cited 4×
“The repeatability can also be calculated on the original (proportion) scale (Browne et al., 2005; Goldstein et al., 2002): RpropM = σ 2 α ·P2/(1 + exp(β0))2 σ 2α ·P2/(1 + exp(β0))2 + ω·P (1 − P ) , (16) P = exp(β0) 1 + exp(β0) (17) where RpropM is the proportion-scale repeatability estimated from…”
BackgroundMethodInfluentialExtending the Linear Model with R
Faraway, J. J. (2016) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315382722 · Cited by 857
Repeatability and heritability of exploratory behaviour in great tits from the wild
Dingemanse, N. J., Both, C., Drent, P. J., van Oers, K., & van Noordwijk, A. J. (2002) · Animal Behaviour · doi:10.1006/anbe.2002.2006 · Cited by 853
Hadfield, J. D., & Nakagawa, S. (2010) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2009.01915.x · Cited by 829
Growth‐mortality tradeoffs and ‘personality traits’ in animals
Stamps, J. A. (2007) · Ecology Letters · doi:10.1111/j.1461-0248.2007.01034.x · Cited by 801
Mccarthy, M. A. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511802454 · Cited by 758
Dealing with overdispersed count data in applied ecology
Richards, S. A. (2007) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2007.01377.x · Cited by 673
Phenotypic plasticity : functional and conceptual approaches
DeWitt, T. J., & Scheiner, S. M. (2004) · Cited by 578
A method for computing profile-likelihood-based confidence intervals
Venzon, D. J., & Moolgavkar, S. H. (1988) · Applied Statistics · doi:10.2307/2347496 · Cited by 565
(2004) · doi:10.1093/oso/9780195138962.001.0001 · Cited by 559
Repeatability: Its role in evolutionary studies of mating behavior
Boake, C. R. B. (1989) · Evolutionary Ecology · doi:10.1007/bf02270919 · Cited by 529
Variance Partitioning in Multilevel Logistic Models that Exhibit Overdispersion
Browne, W. J., Subramanian, S. V., Jones, K., & Goldstein, H. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.00365.x · Cited by 421
Small within-day increases in temperature affects boldness and alters personality in coral reef fish
Biro, P. A., Beckmann, C., & Stamps, J. A. (2009) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2009.1346 · Cited by 401
Generalized linear models with random effects : unified analysis via h-likelihood
Lee, Y., Nelder, J. A., & Pawitan, Y. (2006) · Thammasat University Digital Collections · Cited by 381
Repeatability estimates do not always set an upper limit to heritability
Dohm, M. R. (2002) · Functional Ecology · doi:10.1046/j.1365-2435.2002.00621.x · Cited by 380
Phenotypic plasticity: Functional and conceptual approaches
Crews, D. P. (2004) · American Journal of Human Biology · doi:10.1002/ajhb.20088 · Cited by 324
Generalized Linear Models with Random Effects
Lee, Y., Nelder, J. A., & Pawitan, Y. (2006) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420011340 · Cited by 324
Donner, A. P. (1986) · International Statistical Review · doi:10.2307/1403259 · Cited by 288
Generalized, Linear, and Mixed Models
Mitchell, L. (2003) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00356 · Cited by 272
Estimating the Generalized Concordance Correlation Coefficient through Variance Components
Carrasco, J. L., & Jover, L. (2003) · Biometrics · doi:10.1111/j.0006-341x.2003.00099.x · Cited by 233
Why h 2 does not always equal V A / V P ?
Wilson, A. J. (2008) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2008.01500.x · Cited by 201
The Reliability of Dichotomous Judgments: Unequal Numbers of Judges per Subject
Fleiss, J. L., & Cuzick, J. M. (1979) · Applied Psychological Measurement · doi:10.1177/014662167900300410 · Cited by 191
Merilä, J., & Sheldon, B. C. (2001) · Springer eBooks · doi:10.1007/978-1-4615-1211-0_4 · Cited by 174
Repeatability of mate choice in the zebra finch: consistency within and between females
Forstmeier, W., & Birkhead, T. R. (2004) · Animal Behaviour · doi:10.1016/j.anbehav.2004.02.007 · Cited by 154
Nakagawa, S., Gillespie, D., Hatchwell, B. J., & Burke, T. A. (2007) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2007.01403.x · Cited by 152
Brommer, J. E., Rattiste, K., & Wilson, A. J. (2008) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2007.0951 · Cited by 134
Changing philosophies and tools for statistical inferences in behavioral ecology
Garamszegi, L. Z., Calhim, S., Dochtermann, N. A., Hegyi, G., Hurd, P. L., Jørgensen, C., Kutsukake, N., Lajeunesse, M. J., Pollard, K. A., Schielzeth, H., et al. (2009) · Behavioral Ecology · doi:10.1093/beheco/arp137 · Cited by 133
Confidence Interval Estimation of the Intraclass Correlation Coefficient for Binary Outcome Data
Zou, G., & Donner, A. P. (2004) · Biometrics · doi:10.1111/j.0006-341x.2004.00232.x · Cited by 116
The Interpretation of Negative Components of Variance
Nelder, J. A. (1954) · Biometrika · doi:10.1093/biomet/41.3-4.544 · Cited by 115
The Concordance Correlation Coefficient for Repeated Measures Estimated by Variance Components
Carrasco, J. L., King, T. S., & Chinchilli, V. M. (2009) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400802527890 · Cited by 111
Quantitative genetics of growth and cryptic evolution of body size in an island population
Wilson, A. J., Pemberton, J. M., Pilkington, J. G., Clutton‐Brock, T. H., Coltman, D. W., & Kruuk, L. E. B. (2006) · Evolutionary Ecology · doi:10.1007/s10682-006-9106-z · Cited by 103
How to Make Models Add Up — A Primer on GLMMs
O'Hara, R. B. (2009) · Annales Zoologici Fennici · doi:10.5735/086.046.0205 · Cited by 99
Repeatability of Daily Field Metabolic Rate in Female Meadow Voles (Microtus pennsylvanicus)
Berteaux, D., Thomas, D. W., Bergeron, J., & Lapierre, H. (1996) · Functional Ecology · doi:10.2307/2390510 · Cited by 96
Bolund, E., Schielzeth, H., & Forstmeier, W. (2008) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2008.1251 · Cited by 94
THE STUDY OF INTERSEXUAL SELECTION USING QUANTITATIVE GENETICS
Bakker, T. C. M. (1999) · Behaviour · doi:10.1163/156853999501748 · Cited by 70
Nakagawa, S., Ockendon, N., Gillespie, D., Hatchwell, B. J., & Burke, T. A. (2007) · Oecologia · doi:10.1007/s00442-007-0765-4 · Cited by 70
Gill, J. (2007) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420010824 · Cited by 59
THE COLLABORATIVE LIPID RESEARCH CLINICS PROGRAM FAMILY STUDY
Namboodiri, K. K., Green, P. P., Kaplan, E. B., Morrison, J. A., Chase, G. A., Elston, R. C., Owen, A., Rifkind, B. M., Glueck, C. J., & Tyroler, H. A. (1984) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a113818 · Cited by 48
Carrasco, J. L. (2010) · Biometrics · doi:10.1111/j.1541-0420.2009.01335.x · Cited by 46
Intrasexual competition in zebra finches, the role of beak colour and body size
Bolund, E., Schielzeth, H., & Forstmeier, W. (2007) · Animal Behaviour · doi:10.1016/j.anbehav.2006.10.032 · Cited by 44
Repeatability of extra‐pair mating in tree swallows
Whittingham, L. A., Dunn, P. O., & Stapleton, M. K. (2006) · Molecular Ecology · doi:10.1111/j.1365-294x.2006.02808.x · Cited by 43
Comparison of Concordance Correlation Coefficient Estimating Approaches with Skewed Data
Carrasco, J. L., Jover, L., King, T. S., & Chinchilli, V. M. (2007) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400701329463 · Cited by 40
Patterns of conspecific brood parasitism in zebra finches
Schielzeth, H., & Bolund, E. (2010) · Animal Behaviour · doi:10.1016/j.anbehav.2010.03.006 · Cited by 36
Concordance correlation coefficient applied to discrete data
Carrasco, J. L., & Jover, L. (2005) · Statistics in Medicine · doi:10.1002/sim.2397 · Cited by 35
The design of complex sexual traits in male barn swallows: associations between signal attributes
Garamszegi, L. Z., Hegyi, G., Heylen, D., Ninni, P., de Lope, F., Eens, M., & Møller, A. P. (2006) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2006.01135.x · Cited by 33
Wilson, W. M., Dufour, D. L., Staten, L. K., Barac‐Nieto, M., Reina, J., & Spurr, G. B. (1999) · American Journal of Human Biology · doi:10.1002/(sici)1520-6300(199911/12)11:6 · Cited by 31
Isberg, S. R., Thomson, P. C., Nicholas, F. W., Barker, S. G., & Moran, C. (2005) · Journal of Animal Breeding and Genetics · doi:10.1111/j.1439-0388.2005.00548.x · Cited by 30
HERITABILITY OF AND EARLY ENVIRONMENT EFFECTS ON VARIATION IN MATING PREFERENCES
Schielzeth, H., Bolund, E., & Forstmeier, W. (2009) · Evolution · doi:10.1111/j.1558-5646.2009.00890.x · Cited by 23
Orthogonality and transformations in variance components models
Solomon, P. J. (1999) · Biometrika · doi:10.1093/biomet/86.2.289 · Cited by 11
Research works printed but not linked: 3
- Becker W. A.(1992).A manual of quantitative genetics 5th edition. Academic Enterprises Pullman WA.
- Faraway J. J., 2006, Extending the linear model
- Snijders T. A. B., 1999, Multilevel analysis: an introduction to basic and advanced multilevel modeling.
Cited by
Sorted by citations
A general and simple method for obtaining R 2 from generalized linear mixed‐effects models
Nakagawa, S., & Schielzeth, H. (2013) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2012.00261.x · Cited by 10,356
Multimodel inference in ecology and evolution: challenges and solutions
Grueber, C. E., Nakagawa, S., Laws, R., & Jamieson, I. G. (2011) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2010.02210.x · Cited by 2,355
A brief introduction to mixed effects modelling and multi-model inference in ecology
Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018) · PeerJ · doi:10.7717/peerj.4794 · Cited by 2,094
rptR: repeatability estimation and variance decomposition by generalized linear mixed‐effects models
Stoffel, M. A., Nakagawa, S., & Schielzeth, H. (2017) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12797 · Cited by 1,779
Metadata
- Authors
- Nakagawa, S., & Schielzeth, H.
- Affiliations
- Max Planck Society (DE)
- Corresponding author
- Nakagawa, Shinichi
- Schielzeth, Holger
- Venue
- Biological Reviews
- Publisher
- Wiley
- Published
- 2010-06-21
- Volume
- 85
- Issue
- 4
- Pages
- 935-956
- ISSN
- 0006-3231 (ISSN-L)
- 1464-7931
- 1469-185X
- OpenAlex
- W2045443942
- Cited by
- 2,584
- References
- 85
- Citations by year
- 2026: 109 · 2025: 165 · 2024: 141 · 2023: 171 · 2022: 235 · 2021: 222 · 2020: 206 · 2019: 197 · 2018: 215 · 2017: 198 · 2016: 183 · 2015: 167 · 2014: 136 · 2013: 107 · 2012: 78
- FWCI
- 8.27
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biological reviews/Biological reviews of the Cambridge Philosophical Society (publishedVersion)
- PubMed (publishedVersion)
- PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Life Sciences › Agricultural and Biological Sciences › Ecology, Evolution, Behavior and Systematics › Animal Behavior and Reproduction
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Life Sciences › Agricultural and Biological Sciences › Ecology, Evolution, Behavior and Systematics › Morphological variations and asymmetry
- Keywords
- repeatability, intraclass correlation coefficient, non-Gaussian data, linear mixed models, measurement error, phenotypic flexibility, statistical significance, parametric bootstrapping, permutation tests, Bayesian approach, narrow-sense heritability, binary data, count data, proportion data
- MeSH terms
- Analysis of Variance, Phenotype, Regression Analysis, Statistics as Topic, Reproducibility of Results, Confidence Intervals, Normal Distribution, Linear Models, Least-Squares Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
