Intraclass correlation – A discussion and demonstration of basic features
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
A re-analysis of intraclass correlation (ICC) theory is presented together with Monte Carlo simulations of ICC probability distributions. A partly revised and simplified theory of the single-score ICC is obtained, together with an alternative and simple recipe for its use in reliability studies. Our main, practical conclusion is that in the analysis of a reliability study it is neither necessary nor convenient to start from an initial choice of a specified statistical model. Rather, one may impartially use all three single-score ICC formulas. A near equality of the three ICC values indicates the absence of bias (systematic error), in which case the classical (one-way random) ICC may be used. A consistency ICC larger than absolute agreement ICC indicates the presence of non-negligible bias; if so, classical ICC is invalid and misleading. An F-test may be used to confirm whether biases are present. From the resulting model (without or with bias) variances and confidence intervals may then be calculated. In presence of bias, both absolute agreement ICC and consistency ICC should be reported, since they give different and complementary information about the reliability of the method. A clinical example with data from the literature is given.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Practical Statistics for Medical Research.
Walter, S. D., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,668
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903 · cited 9×
“The EMS relations are of key importance for understanding the intraclass correlation formulas, but in medical and psychological articles as well as in medical statistics textbooks they are seldom quoted, then only partly, without any derivation and even without giving any references [1,5,6].”
BackgroundMethodInfluentialQUANTIFYING TEST-RETEST RELIABILITY USING THE INTRACLASS CORRELATION COEFFICIENT AND THE SEM
WEIR, J. P. (2005) · Journal of Strength and Conditioning Research · doi:10.1519/00124278-200502000-00038 · Cited by 4,842
Practical Statistics for Medical Research.
Bartko, J. J., & Altman, D. G. (1992) · Journal of the American Statistical Association · doi:10.2307/2290246 · Cited by 4,797
Streiner, D. L., & Norman, G. R. (2008) · doi:10.1093/acprof:oso/9780199231881.001.0001 · Cited by 2,631
Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists.
Nakagawa, S., & Schielzeth, H. (2010) · Biological Reviews · doi:10.1111/j.1469-185x.2010.00141.x · Cited by 2,584 · cited 2×
“It is also sometimes known as a linear or linear mixed-effects model (LMM) [18].”
BackgroundMethod
Cited by
An Examination of the Use of Large Language Models to Aid Analysis of Textual Data
(2024) · doi:10.1177/16094069241231168 · Cited by 227
Radiomics feature reproducibility under inter-rater variability in segmentations of CT images
Haarburger, C., Müller-Franzes, G., Weninger, L., Kuhl, C., Truhn, D., & Merhof, D. (2020) · Scientific Reports · doi:10.1038/s41598-020-69534-6 · Cited by 139
“In order to assess feature robustness across segmentations, we evaluated the intraclass correlation coefficient ICC ( 1, 1 ) 33, 34 based on a one-way random model.”
MethodBui, N. M., & Barrot, J. S. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12891-w · Cited by 113
“The following ICC interpretation was applied (Liljequist et al., 2019): <0.50 – poor; 0.50 to 0.75 – moderate; 0.75 to 0.90 – good; above 0.90 – excellent.”
InfluentialZhang, X., Song, J., Zhang, Y., Wen, B., Dai, L., Xi, R., Wu, Q., Li, Y., Luo, X., Lan, X., et al. (2023) · Journal of Hepatology · doi:10.1016/j.jhep.2022.10.030 · Cited by 62
Data-driven studies in face identity processing rely on the quality of the tests and data sets
Bobak, A. K., Jones, A. L., Hilker, Z., Mestry, N., Bate, S., & Hancock, P. J. B. (2023) · Cortex · doi:10.1016/j.cortex.2023.05.018 · Cited by 23
ChatGPT identifies gender disparities in scientific peer review
Verharen, J. P. (2023) · eLife · doi:10.7554/elife.90230 · Cited by 16
Garbarino, S., & Bragazzi, N. L. (2024) · Journal of Sleep Research · doi:10.1111/jsr.14210 · Cited by 15
Wang, J., Chen, Z., Zhang, H., Li, W., Li, K., Deng, M., & Zou, Y. (2025) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-025-07450-1 · Cited by 9
SDoH Impact on Periodontal Disease Using Machine Learning and Dental Records
Patel, J., Badi, M., Katiyar, R., Ogwo, C., Wiener, R. C., Tiwari, T., Sambamoorthi, U., & Folks, T. (2025) · Journal of Dental Research · doi:10.1177/00220345251328968 · Cited by 8
“…the interclass correlation coefficient (ICC) to examine the impact of clustered data on our inferences when linking patient-level data with ZIP codes and found a low ICC (0.02), indicating minimal clustering (only 2% of PD variance attributable to ZIP code differences) (Liljequist et al. 2019).”
Influential
Metadata
- Authors
- Liljequist, D., Elfving, B., & Skavberg Roaldsen, K.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-07-22
- Volume
- 14
- Issue
- 7
- Pages
- e0219854
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2963121203
- Cited by
- 905
- References
- 32
- FWCI
- 36.72
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.