The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Intraclass correlation – A discussion and demonstration of basic features
Open accessAbstract
A re-analysis of intraclass correlation (ICC) theory is presented together with Monte Carlo simulations of ICC probability distributions. A partly revised and simplified theory of the single-score ICC is obtained, together with an alternative and simple recipe for its use in reliability studies. Our main, practical conclusion is that in the analysis of a reliability study it is neither necessary nor convenient to start from an initial choice of a specified statistical model. Rather, one may impartially use all three single-score ICC formulas. A near equality of the three ICC values indicates the absence of bias (systematic error), in which case the classical (one-way random) ICC may be used. A consistency ICC larger than absolute agreement ICC indicates the presence of non-negligible bias; if so, classical ICC is invalid and misleading. An F-test may be used to confirm whether biases are present. From the resulting model (without or with bias) variances and confidence intervals may then be calculated. In presence of bias, both absolute agreement ICC and consistency ICC should be reported, since they give different and complementary information about the reliability of the method. A clinical example with data from the literature is given.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Practical Statistics for Medical Research.
Walter, S. D., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,668
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903
QUANTIFYING TEST-RETEST RELIABILITY USING THE INTRACLASS CORRELATION COEFFICIENT AND THE SEM
WEIR, J. P. (2005) · Journal of Strength and Conditioning Research · doi:10.1519/00124278-200502000-00038 · Cited by 4,842
Practical Statistics for Medical Research.
Bartko, J. J., & Altman, D. G. (1992) · Journal of the American Statistical Association · doi:10.2307/2290246 · Cited by 4,797
Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists.
Nakagawa, S., & Schielzeth, H. (2010) · Biological Reviews · doi:10.1111/j.1469-185x.2010.00141.x · Cited by 2,584
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,299
When to use agreement versus reliability measures
de Vet, H. C. W., Terwee, C. B., Knol, D. L., & Bouter, L. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.10.015 · Cited by 1,786
On various intraclass correlation reliability coefficients.
Bartko, J. J. (1976) · Psychological Bulletin · doi:10.1037//0033-2909.83.5.762 · Cited by 825
A critical discussion of intraclass correlation coefficients
Müller, R., & Büttner, P. (1994) · Statistics in Medicine · doi:10.1002/sim.4780132310 · Cited by 714
Cicchetti, D. V., Shoinralter, D., & Tyrer, P. J. (1985) · Applied Psychological Measurement · doi:10.1177/014662168500900103 · Cited by 291
Intraclass correlations: There's more there than meets the eye.
Lahey, M. A., Downey, R. G., & Saal, F. E. (1983) · Psychological Bulletin · doi:10.1037//0033-2909.93.3.586 · Cited by 222
Reliability and validity of the French version of the global physical activity questionnaire
Rivière, F., Widad, F. Z., Speyer, É., Erpelding, M., Escalon, H., & Vuillemin, A. (2016) · Journal of sport and health science/Journal of Sport and Health Science · doi:10.1016/j.jshs.2016.08.004 · Cited by 125
Testing the equality of dependent intraclass correlation coefficients
Donner, A. P., & Zou, G. (2002) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00324 · Cited by 116
Effective number of subjects and number of raters for inter‐rater reliability studies
Saito, Y., Sozu, T., Hamada, C., & Yoshimura, I. (2005) · Statistics in Medicine · doi:10.1002/sim.2294 · Cited by 96
Reliability of EMG spectral parameters in repeated measurements of back muscle fatigue
Elfving, B., Németh, G., Arvidsson, I., et al. (1999) · Journal of Electromyography and Kinesiology · doi:10.1016/s1050-6411(98)00049-2 · Cited by 93
Self-citationLöfgren, N., Lenholm, E., Conradsson, D. M., Ståhle, A., & Franzén, E. (2014) · BMC Neurology · doi:10.1186/s12883-014-0235-7 · Cited by 87
Rennie, L., Löfgren, N., Moe‐Nilssen, R., Opheim, A. I., Dietrichs, E., & Franzén, E. (2018) · Gait & Posture · doi:10.1016/j.gaitpost.2018.04.011 · Cited by 69
Reliability of radiographic measurements for acute distal radius fractures
Watson, N. J., Asadollahi, S., Parrish, F., Ridgway, J., Tran, P., & Keating, J. L. (2016) · BMC Medical Imaging · doi:10.1186/s12880-016-0147-7 · Cited by 42
The Estimation of Reliability When Several Trials are Available
Alexander, H. W. (1947) · Psychometrika · doi:10.1007/bf02295990 · Cited by 39
Werner, I. M., Ernst, M. J., Treleaven, J. M., et al. (2018) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-018-2287-0 · Cited by 39
Shanbehzadeh, S., Salavati, M., Tavahomi, M., Khatibi, A., Talebian, S., & Kalantari, K. K. (2017) · Spine · doi:10.1097/brs.0000000000002149 · Cited by 30
Frykholm, E., Géphine, S., Saey, D., van Hees, H. W. H., Lemson, A., Klijn, P. H. C., Maltais, F., & Nyberg, A. (2018) · Chronic Respiratory Disease · doi:10.1177/1479973118816497 · Cited by 28
Roaldsen, K. S., Måøy, Å. B., Jørgensen, V., & Stanghelle, J. K. (2015) · Journal of Spinal Cord Medicine · doi:10.1080/10790268.2015.1119965 · Cited by 22
Test-retest reliability of knee extensors endurance test with elastic resistance
Lopes, J. S. S., Micheletti, J. K., Machado, A. F., Souto, L. R., de Lima, H. P., Vanderlei, F. M., Netto, J., & Pastre, C. M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0203259 · Cited by 20
Brandt, M., Andersen, L. L., Samani, A., Jakobsen, M. D., & Madeleine, P. (2017) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-017-1872-y · Cited by 18
Reproducibility of manual pressure force on provocation of the sacroiliac joint
Levin, U., Nilsson-Wikmar, L. B., Stenstruöm, C. H., & Lundeberg, T. (1998) · Physiotherapy Research International · doi:10.1002/pri.118 · Cited by 15
Bohman, T., Tegern, M., Halvarsson, A., Broman, L., & Larsson, H. (2018) · PLoS ONE · doi:10.1371/journal.pone.0209419 · Cited by 9
Cited by
An Examination of the Use of Large Language Models to Aid Analysis of Textual Data
(2024) · doi:10.1177/16094069241231168 · Cited by 227
Radiomics feature reproducibility under inter-rater variability in segmentations of CT images
Haarburger, C., Müller-Franzes, G., Weninger, L., Kuhl, C., Truhn, D., & Merhof, D. (2020) · Scientific Reports · doi:10.1038/s41598-020-69534-6 · Cited by 139
Bui, N. M., & Barrot, J. S. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12891-w · Cited by 113
“The following ICC interpretation was applied (Liljequist et al., 2019): <0.50 – poor; 0.50 to 0.75 – moderate; 0.75 to 0.90 – good; above 0.90 – excellent.”
InfluentialZhang, X., Song, J., Zhang, Y., Wen, B., Dai, L., Xi, R., Wu, Q., Li, Y., Luo, X., Lan, X., et al. (2023) · Journal of Hepatology · doi:10.1016/j.jhep.2022.10.030 · Cited by 62
Data-driven studies in face identity processing rely on the quality of the tests and data sets
Bobak, A. K., Jones, A. L., Hilker, Z., Mestry, N., Bate, S., & Hancock, P. J. B. (2023) · Cortex · doi:10.1016/j.cortex.2023.05.018 · Cited by 23
Metadata
- Authors
- Liljequist, D., Elfving, B., & Skavberg Roaldsen, K.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-07-22
- Volume
- 14
- Issue
- 7
- Pages
- e0219854
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2963121203
- Cited by
- 905
- References
- 31
- FWCI
- 36.72
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.