The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
A critical discussion of intraclass correlation coefficients
Abstract
In general, intraclass correlation coefficients (ICC's) are designed to assess consistency or conformity between two or more quantitative measurements. They are claimed to handle a wide range of problems, including questions of reliability, reproducibility and validity. It is shown that care must be taken in choosing a suitable ICC with respect to the underlying sampling theory. For this purpose a decision tree is developed. It may be used to choose a coefficient which is appropriate for a specific study setting. We demonstrate that different ICC's may result in quite different values for the same data set, even under the same sampling theory. Other general limitations of ICC's are also addressed. Potential alternatives are presented and discussed, and some recommendations are given for the use of an appropriate method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
Statistical Methods for Rates and Proportions.
Everitt, B. S., & Fleiss, J. L. (1981) · Biometrics · doi:10.2307/2530193 · Cited by 8,990
A concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785
The Advanced Theory of Statistics.
Clarke, G. M., Kendall, M., & Stuart, A. (1978) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344782 · Cited by 6,658
Measurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300
Fleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354
Statistical Methods for Research Workers.
I., L., & Fisher, R. A. (1926) · Journal of the Royal Statistical Society · doi:10.2307/2341488 · Cited by 2,128
Clinical biostatistics: LIV. The biostatistics of concordance
Kramer, M. S., & Feinstein, A. R. (1981) · Clinical Pharmacology and Therapeutics · doi:10.1038/clpt.1981.18 · Cited by 1,087
Cited by
DPABI: Data Processing & Analysis for (Resting-State) Brain Imaging
Yan, C.-G., Wang, X.-D., Zuo, X.-N., & Zang, Y.-F. (2016) · Neuroinformatics · doi:10.1007/s12021-016-9299-4 · Cited by 3,853
Atkinson, G., & Nevill, A. M. (1998) · Sports Medicine · doi:10.2165/00007256-199826040-00002 · Cited by 3,779
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.002 · Cited by 2,174
Consensus measurement in Delphi studies
von der Gracht, H. A. (2012) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2012.04.013 · Cited by 1,831
Ukoumunne, Gulliford, Chinn, Sterne, & Burney (1999) · Health Technology Assessment · doi:10.3310/hta3050 · Cited by 1,210
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audige, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.01.016 · Cited by 997
Intraclass correlation – A discussion and demonstration of basic features
Liljequist, D., Elfving, B., & Skavberg Roaldsen, K. (2019) · PLOS ONE · doi:10.1371/journal.pone.0219854 · Cited by 905
Shaha, J. S., Cook, J. B., Song, D. J., Rowles, D. J., Bottoni, C. R., Shaha, S. H., & Tokish, J. M. (2015) · The American Journal of Sports Medicine · doi:10.1177/0363546515578250 · Cited by 688
Noble, S., Scheinost, D., & Constable, R. T. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116157 · Cited by 580
How reliable are the results from functional magnetic resonance imaging?
Bennett, C. M., & Miller, M. B. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05446.x · Cited by 577
Statistical guidelines for the analysis of data obtained from one or both eyes
Armstrong, R. A. (2013) · Ophthalmic and Physiological Optics · doi:10.1111/opo.12009 · Cited by 522
A Note On "A Concordance Correlation Coefficient to Evaluate Reproducibility"
Nickerson, C. A. E. (1997) · Biometrics · doi:10.2307/2533516 · Cited by 510
Mallidi, S., Larson, T., Tam, J., Joshi, P. P., Karpiouk, A., Sokolov, K., & Emelianov, S. (2009) · Nano Letters · doi:10.1021/nl802929u · Cited by 446
Braun, U., Plichta, M. M., Esslinger, C., Sauer, C., Haddad, L., Grimm, O., Mier, D., Mohnke, S., Heinz, A., Erk, S., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.044 · Cited by 424
Novel Measurement Technique of the Tibial Slope on Conventional MRI
Hudek, R., Schmutz, S., Regenfelder, F., Fuchs, B., & Koch, P. P. (2009) · Clinical Orthopaedics & Related Research · doi:10.1007/s11999-009-0711-3 · Cited by 387
Gau, S. S., Shang, C., Liu, S., Lin, C., Swanson, J. M., Liu, Y., & Tu, C. (2008) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.237 · Cited by 375
Noble, S., Spann, M. N., Tokoglu, F., Shen, X., Constable, R. T., & Scheinost, D. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx230 · Cited by 377
Reexamination of Validity and Reliability of the CSA Monitor in Walking and Running
BRAGE, S., WEDDERKOPP, N., FRANKS, P. W., BO ANDERSEN, L., & FROBERG, K. (2003) · Medicine & Science in Sports & Exercise · doi:10.1249/01.mss.0000079078.62035.ec · Cited by 347
Adaptación cultural y validación de cuestionarios de salud: revisión y recomendaciones metodológicas
Ramada-Rodilla, J. M., Serra-Pujadas, C., & Delclós-Clanchet, G. L. (2013) · Salud Pública de México · doi:10.1590/s0036-36342013000100009 · Cited by 327
An Overview on Assessing Agreement with Continuous Measurements
Barnhart, H. X., Haber, M. J., & Lin, L. I. (2007) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400701376480 · Cited by 319
The reliability of short-term measurements of heart rate variability
Sandercock, G. R. H., Bromley, P. D., & Brodie, D. A. (2005) · International Journal of Cardiology · doi:10.1016/j.ijcard.2004.09.013 · Cited by 307
Measurement error in geometric morphometrics
Fruciano, C. (2016) · Development Genes and Evolution · doi:10.1007/s00427-016-0537-4 · Cited by 301
Deep Learning Hybrid Method to Automatically Diagnose Periodontal Bone Loss and Stage Periodontitis
Chang, H.-J., Lee, S.-J., Yong, T.-H., Shin, N.-Y., Jang, B.-G., Kim, J.-E., Huh, K.-H., Lee, S.-S., Heo, M.-S., Choi, S.-C., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-64509-z · Cited by 290
Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery.
Plichta, M. M., Schwarz, A. J., Grimm, O., Morgen, K., Mier, D., Haddad, L., Gerdes, A. B. M., Sauer, C., Tost, H., Esslinger, C., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.129 · Cited by 287
Test-retest reliability of trunk accelerometry during standing and walking
Moe-Nilssen, R. (1998) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(98)90231-3 · Cited by 271
Cacciola, J. S., Alterman, A. I., McLellan, A. T., Lin, Y.-T., & Lynch, K. G. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.09.002 · Cited by 271
Wenger, E., Mårtensson, J., Noack, H., Bodammer, N. C., Kühn, S., Schaefer, S., Heinze, H., Düzel, E., Bäckman, L., Lindenberger, U., et al. (2014) · Human Brain Mapping · doi:10.1002/hbm.22473 · Cited by 151
Reliability of ventilatory parameters during cycle ergometry in multicentre trials in COPD
O’Donnell, D. E., Travers, J., Webb, K. A., He, Z., Lam, Y.-M., Hamilton, A., Kesten, S., Maltais, F., & Magnussen, H. (2009) · European Respiratory Journal · doi:10.1183/09031936.00168708 · Cited by 113
Chien, W. T., Mui, J., Gray, R., & Cheung, E. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0744-6 · Cited by 48
Metadata
- Authors
- Müller, R., & Büttner, P.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 1994-12-15
- Volume
- 13
- Issue
- 23-24
- Pages
- 2465-2476
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2137481238
- Cited by
- 714
- References
- 22
- FWCI
- 5.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.