The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 11,701–11,800 of 23,881
Defense Mechanism Card Pull in TAT Stories
Cramer, P. (2016) · Journal of Personality Assessment · doi:10.1080/00223891.2016.1207080 · Cited by 17
Devocht, E. M. J., Janssen, A. M. L., Chalupper, J., Stokroos, R. J., & George, E. L. J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0160829 · Cited by 18
An Ecological Study of Anterior Cruciate Ligament Reconstruction, Part 1
McGrath, T. M., Waddington, G. S., Scarvell, J. M., et al. (2016) · Orthopaedic Journal of Sports Medicine · doi:10.1177/2325967116672208 · Cited by 17
Solomonov, N., Kuprian, N., Zilcha‐Mano, S., Gorman, B. S., & Barber, J. P. (2015) · Journal of Psychotherapy Integration · doi:10.1037/int0000021 · Cited by 17
Examining Interobserver Reliability of Metric and Morphoscopic Characteristics of the Mandible
Byrnes, J. F., Kenyhercz, M. W., & Berg, G. E. (2016) · Journal of Forensic Sciences · doi:10.1111/1556-4029.13349 · Cited by 17
The Longitudinal Effects of Parenting on Adaptive Behavior in Children with Fragile X Syndrome
Warren, S. F., Brady, N. C., Fleming, K., & Hahn, L. J. (2017) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-016-2999-7 · Cited by 17
Fonticoba, T. D. D., Santos‐García, D., & Arribi, M. M. (2017) · Neurología · doi:10.1016/j.nrl.2017.03.005 · Cited by 17
Jugular Anomalies in Multiple Sclerosis Are Associated with Increased Collateral Venous Flow
Sethi, S. K., Daugherty, A. M., Gadda, G., et al. (2017) · American Journal of Neuroradiology · doi:10.3174/ajnr.a5219 · Cited by 17
Validity and Reliability of a New Device (WIMU®) for Measuring Hamstring Muscle Extensibility
Muyor, J. M. (2017) · International Journal of Sports Medicine · doi:10.1055/s-0043-108998 · Cited by 17
To search or to like: Mapping fixations to differentiate two forms of incidental scene memory
Choe, K. W., Kardan, O., Kotabe, H., Henderson, J. M., & Berman, M. G. (2017) · Journal of Vision · doi:10.1167/17.12.8 · Cited by 17
Bérubé, M., Czyrnyj, C. S., & McLean, L. (2018) · Neurourology and Urodynamics · doi:10.1002/nau.23575 · Cited by 17
Tahmasebi, N., Shafie, B., Karimi, H., & Mazaheri, M. (2018) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2018.04.005 · Cited by 17
Chan, P. H. Y., Chan, M. H. M., Li, A. M., Cheung, R. C. K., Yu, X., & Lam, H. S. (2018) · Environmental Research · doi:10.1016/j.envres.2018.06.033 · Cited by 17
Munch, K. L., Wapstra, E., Thomas, S., Fisher, M., & Sinn, D. L. (2019) · Ethology · doi:10.1111/eth.12846 · Cited by 17
The Child Abuse Risk Evaluation Dutch Version (CARE-NL): A retrospective validation study
de Ruiter, C., Hildebrand, M., & van der Hoorn, S. (2019) · Journal of Family Trauma Child Custody & Child Development · doi:10.1080/15379418.2019.1699488 · Cited by 17
Effects of Adventitious Acute Vocal Trauma: Relative Fundamental Frequency and Listener Perception
Murray, E. S. H., Hands, G. L., Calabrese, C. R., et al. (2015) · Journal of Voice · doi:10.1016/j.jvoice.2015.04.005 · Cited by 17
A non-invasive assessment of cardiopulmonary hemodynamics with MRI in pulmonary hypertension
Bane, O., Shah, S. J., Cuttica, M. J., et al. (2015) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2015.08.005 · Cited by 17
Reliability of a device designed to measure ankle mobility
Harvey, L. A., Byak, A., Ostrovskaya, M., et al. (2003) · Spinal Cord · doi:10.1038/sj.sc.3101486 · Cited by 17
Impact of the National Practitioner Data Bank on Resolution of Malpractice Claims
Waters, T. M., Studdert, D. M., Brennan, T. A., Thomas, E. J., Almagor, O., Mancewicz, M., & Budetti, P. P. (2003) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.5034/inquiryjrnl_40.3.283 · Cited by 17
Theoretical and methodological development in the study of pathological voice quality
Gerratt, B. R., & Kreiman, J. E. (2000) · Journal of Phonetics · doi:10.1006/jpho.2000.0120 · Cited by 17
Situational feature recognition in schizophrenic outpatients
Corrigan, P. W., Garman, A. N., & Nelson, D. R. (1996) · Psychiatry Research · doi:10.1016/0165-1781(96)02798-9 · Cited by 17
Effects of Dichotomous Lexical Stimuli in Concept Generation
Chiu, I., & Shu, L. H. (2008) · doi:10.1115/detc2008-49372 · Cited by 17
Reliability of the Australian Therapy Outcome Measures for Occupational Therapy Self‐care scale
Scott, F., Unsworth, C. A., Fricke, J., et al. (2006) · Australian Occupational Therapy Journal · doi:10.1111/j.1440-1630.2006.00584.x · Cited by 17
Physical Attractiveness in Schizophrenia
Penn, D. L., Mueser, K. T., & Doonan, R. J. (1997) · Behavior Modification · doi:10.1177/01454455970211004 · Cited by 17
Evaluating Laryngotracheal Stenosis in a Canine Model with Virtual Bronchoscopy
Davros, W. J., Eliachar, I., Eliashar, R., et al. (2000) · Annals of Otology Rhinology & Laryngology · doi:10.1177/000348940010901003 · Cited by 17
Validation of the Symptoms of Illness Checklist (SIC) as a Tool for Health Psychology Research
Stowell, J. R., Hedges, D. W., Ghambaryan, A., et al. (2009) · Journal of Health Psychology · doi:10.1177/1359105308097947 · Cited by 17
Tickle‐Degnen, L., & Coster, W. J. (1995) · The Occupational Therapy Journal of Research · doi:10.1177/153944929501500205 · Cited by 17
SCHNEIDERAXMANN, T., Kamer, T., MORONI, M., et al. (2005) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2005.04.009 · Cited by 17
The Reliability and Validity of the Tekdyne Hand Dynamometer
Stephens, J. L., Pratt, N. E., & Parks, B. G. (1996) · Journal of Hand Therapy · doi:10.1016/s0894-1130(96)80006-8 · Cited by 17
Bonél, H., Jäger, L., Frei, K. A., et al. (2005) · American Journal of Roentgenology · doi:10.2214/ajr.185.3.01850647 · Cited by 17
Klein, C., Milrod, B. L., Busch, F. N., Levy, K. N., & Shapiro, T. (2003) · Psychoanalytic Inquiry · doi:10.1080/07351692309349035 · Cited by 17
Harter, S. L., Hart, C. C., & Harter, G. W. (1999) · Archives of Clinical Neuropsychology · doi:10.1016/s0887-6177(98)00033-x · Cited by 17
Evaluation of the multi-level Roland-Morris disability questionnaire
Chansirinukor, W., Maher, C. G., & Latimer, J. (2004) · Physiotherapy Theory and Practice · doi:10.1080/09593980490425049 · Cited by 17
Reliability and reactivity of three new functional assessment measures
Matheson, L. N., Rogers, L. C., Kaskutas, V., et al. (2002) · Work · doi:10.3233/wor-2002-00201 · Cited by 17
The Statistical Power of a Decade of Social Work Education Research
Orme, J. G. E., & Tolman, R. (1986) · Social Service Review · doi:10.1086/644403 · Cited by 16
The Hillside Akathisia Scale: a reliability comparison of the English and German versions
Fleischhacker, W. W., Miller, C. H., Schett, P., et al. (1991) · Psychopharmacology · doi:10.1007/bf02316879 · Cited by 17
Baker, N. A., Cook, J. R., & Redfern, M. S. (2008) · Applied Ergonomics · doi:10.1016/j.apergo.2007.12.008 · Cited by 17
Sloan, R. P., Shapiro, P. A., Bagiella, E., Myers, M. M., Bigger, J. T., Steinman, R. C., & Gorman, J. M. (1994) · Biological Psychology · doi:10.1016/0301-0511(94)90034-5 · Cited by 17
Interrater/Test Reliability System (ITRS)
Abedi, J. (1996) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3104_1 · Cited by 17
Gao, F., Black, S. E., Leibovitch, F. S., Callen, D. J., Rockel, C. P., & Szalai, J. P. (2003) · Neurobiology of Aging · doi:10.1016/s0197-4580(03)00121-0 · Cited by 17
Long‐term changes after brief dynamic psychotherapy: symptomatic versus dynamic assessments
Høglend, P. A., Sørlie, T., Sørbye, Ø., et al. (1992) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1992.tb03246.x · Cited by 17
Technical Variability in Laboratory Data
Vineis, P., Schulte, P. A., & Vogt, R. F. (1993) · Elsevier eBooks · doi:10.1016/b978-0-08-092566-0.50008-2 · Cited by 17
Using the Data Agreement Criterion to Rank Experts' Beliefs
Veen, D., Stoel, D., Schalken, N., Mulder, K., van de Schoot, R., Sciences, M. A. S. F. T. B. A. S., & Hoijtink, L. (2016) · Utrecht University Repository (Utrecht University) · Cited by 17
Gordon, R. M., & Baron, E. M. (2010) · Springer eBooks · doi:10.1007/978-0-387-79948-3_2013 · Cited by 18
Portable Inertial Motion Unit for Continuous Assessment of In-shoe Foot Movement
Mifsud, N. L., Kristensen, N. H., Villumsen, M., Hansen, J., & Kersting, U. G. (2014) · Procedia Engineering · doi:10.1016/j.proeng.2014.06.035 · Cited by 16
Budeanu, R. G., Broemmer, C., Budeanu, A. R., et al. (2022) · Tomography · doi:10.3390/tomography8050201 · Cited by 16
Caronni, A., Picardi, M., Scarano, S., Rota, V., Guidali, G., Bolognini, N., & Corbo, M. (2024) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-024-01339-4 · Cited by 18
Reisinger, D. L., Goodwin, M. S., Horn, P. S., Schmitt, L., Coffman, M. C., & Shaffer, R. C. (2024) · Scientific Reports · doi:10.1038/s41598-024-66084-z · Cited by 16
Jiang, J., Chen, J., Lin, Z., Tang, X., & Hu, Z. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1466426 · Cited by 20
Köse, N., Şimşek, E., & Demir, M. C. (2025) · Current Psychology · doi:10.1007/s12144-025-07418-6 · Cited by 23
Yavuz, B. Ş., Ekmekcioglu, O., & Ankaralı, H. Ç. (2025) · International Journal of Legal Medicine · doi:10.1007/s00414-025-03452-y · Cited by 16
The prevalence of potentially preventable deaths in an acute care hospital
Kobewka, D. M., van Walraven, C., Taljaard, M., Ronksley, P. E., & Forster, A. J. (2017) · Medicine · doi:10.1097/md.0000000000006162 · Cited by 16
Impact of rapport on neuropsychological test performance
Barnett, M. D., Parsons, T. D., Reynolds, B. L., & Bedford, L. A. (2017) · Applied Neuropsychology Adult · doi:10.1080/23279095.2017.1293671 · Cited by 16
Bettens, K., Alighieri, C., Bruneel, L., Meulemeester, L. D., & Lierde, K. M. V. (2020) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2020.105991 · Cited by 16
On the design and validation of a rubric for the evaluation of performance in a musical contest
Álvarez-Díaz, M., Muñiz-Bascón, L. M., Soria-Alemany, A., Veintimilla-Bonet, A., & Fernández-Alonso, R. (2020) · International Journal of Music Education · doi:10.1177/0255761420936443 · Cited by 16
Haji‐Valizadeh, H., Shen, D., Avery, R. J., Serhal, A., Schiffers, F., Katsaggelos, A. K., Cossairt, O. S., & Kim, D. J. (2020) · Radiology Cardiothoracic Imaging · doi:10.1148/ryct.2020190205 · Cited by 16
Teixeira, L. F., Diz, J. B. M., da Silva, S. L. A., Viana, J. U., Dias, J. M. D., Pereira, L. S. M., & Dias, R. C. (2020) · Sao Paulo Medical Journal · doi:10.1590/1516-3180.2019.0542.r2.16042020 · Cited by 17
Schmartz, D., Chénard, L., Baumann, C., et al. (2020) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-020-00560-5 · Cited by 16
Wang, G., Wu, L., Su, H., et al. (2020) · Arthritis & Rheumatology · doi:10.1002/art.41506 · Cited by 16
Empirical Test Guidelines for Content Validity: Wash, Rinse, and Repeat until Clean
Schmitz, K., & Storey, V. C. (2020) · Communications of the Association for Information Systems · doi:10.17705/1cais.04736 · Cited by 17
Kazdal, D. N., Rempel, E., de Oliveira, C. M. M., Allgäuer, M., Harms, A., Singer, K., Kohlwes, E., Ormanns, S., Fink, L., Kriegsmann, J., et al. (2021) · Translational Lung Cancer Research · doi:10.21037/tlcr-20-1168 · Cited by 16
Exploring the Effects of Interruptions in Different Phases of Complex Decision-Making Tasks
Wu, M., Gao, Q., & Liu, Y. (2021) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211018882 · Cited by 19
Sex and posture dependence of neck muscle size-strength relationships
Reddy, C. P., Zhou, Y. M., Wan, B., & Zhang, X. (2021) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2021.110660 · Cited by 16
Sayegh, E. T., & Matzkin, E. (2021) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001948 · Cited by 16
The crescent sign—a predictor of hip instability in magnetic resonance arthrography
Zurmühle, C. A., Kuner, V., McInnes, J., Pfluger, D. H., & Beck, M. (2021) · Journal of Hip Preservation Surgery · doi:10.1093/jhps/hnab067 · Cited by 16
Test-retest reliability of the 5-minute psychomotor vigilance task in working-aged females
Thompson, B. J., Shugart, C., Dennison, K., & Louder, T. J. (2021) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2021.109379 · Cited by 16
Amemiya, S., Takao, H., Watanabe, Y., et al. (2021) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.27996 · Cited by 16
Morgan, S. J., Askew, R. L., & Hafner, B. J. (2021) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2021.10.008 · Cited by 16
Interrater reliability of quantitative ultrasound measures of muscle in critically ill patients
Baston, C. M., Gellhorn, A. C., Hough, C. L., et al. (2022) · PM&R · doi:10.1002/pmrj.12758 · Cited by 16
Fischer, T., Schraivogel, S., Caversaccio, M. D., & Wimmer, W. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.856219 · Cited by 16
National survey of periodontal diseases in adolescents and young adults in Morocco
Kissa, J., Albandar, J. M., Houari, B. E., et al. (2022) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13613 · Cited by 16
Neural Network Prediction of Slurry Erosion Wear of Ni-WC Coated Stainless Steel 420
Kumar, S., Chandra, S. K., Dixit, S., et al. (2022) · Metals · doi:10.3390/met12050706 · Cited by 16
Jaeger, B., Evans, A. M., Stel, M., & van Beest, I. (2022) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2022.104288 · Cited by 18
Validity and reliability of the Danish version of the Pittsburgh Sleep Quality Index
Ørskov, P. T., & Norup, A. (2022) · Scandinavian Journal of Public Health · doi:10.1177/14034948221100099 · Cited by 18
Abdollahipour, R., Land, W. M., Valtr, L., Banátová, K., & Janura, M. (2022) · International Journal of Sport and Exercise Psychology · doi:10.1080/1612197x.2022.2098356 · Cited by 20
Tirrito, F., Cozzi, F., Bonaldi, M., et al. (2022) · Journal of Veterinary Medical Science · doi:10.1292/jvms.22-0134 · Cited by 16
Zheng, Q., Zhang, Y., Li, H., Tong, X., & Ouyang, M. (2022) · European Radiology · doi:10.1007/s00330-022-09081-y · Cited by 16
Automated Scoring of Sleep and Associated Events
Anderer, P., Ross, M., Cerny, A., & Shaw, E. (2022) · Advances in experimental medicine and biology · doi:10.1007/978-3-031-06413-5_7 · Cited by 16
Validity and reliability of the Output sport device for assessing drop jump performance
Montoro-Bombú, R., Field, A., Santos, A. M., et al. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.1015526 · Cited by 16
Arslanova, I., Galvez-Pol, A., Kilner, J. M., Finotti, G., & Manos Tsakiris, A. (2022) · Affective Science · doi:10.1007/s42761-022-00151-4 · Cited by 21
Identification of Critical Success and Risk Factors for Public–Private Partnership Highway Projects
Fathi, M., & Shrestha, P. P. (2022) · Journal of Legal Affairs and Dispute Resolution in Engineering and Construction · doi:10.1061/(asce)la.1943-4170.0000587 · Cited by 16
Barbera, F., Hasso, T., & Schwarz, T. V. (2022) · Journal of Family Business Management · doi:10.1108/jfbm-06-2022-0076 · Cited by 16
Joustra, V., Yim, A. Y. F. L., Hageman, I., Levin, E., Adams, A. T., Satsangi, J. J., de Jonge, W. J., Henneman, P., & D’Haens, G. (2022) · Cellular and Molecular Gastroenterology and Hepatology · doi:10.1016/j.jcmgh.2022.12.011 · Cited by 16
The Spanish Version of the International Index of Erectile Function: Adaptation and Validation
Díaz-Mohedo, E., Sánchez, A. M., Santamaría, F. C., García, E. M., Hernández, S. H., & Hita‐Contreras, F. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20031830 · Cited by 16
Tomacheuski, R. M., de Oliveira, A. R., Trindade, P. H. E., de Oliveira, F. A., Candido, C. P., Neto, F. J. T., Steagall, P. V. M., & Luna, S. P. L. (2023) · Animals · doi:10.3390/ani13030364 · Cited by 15
Validity and reliability of VO2-max testing in persons with Parkinson's disease
Bonde-Jensen, F., Dalgas, U., Brincks, J., & Langeskov‐Christensen, M. (2023) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2023.105324 · Cited by 18
Eveleigh, K. J., Deluzio, K. J., Scott, S. H., & Laende, E. K. (2022) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2023.111556 · Cited by 16
Nyman, T., Rhén, I.-M., Johansson, P. J., Eliasson, K., Kjellberg, K., Lindberg, P., Fan, X., & Forsman, M. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20085505 · Cited by 16
Cummock, J. S., Wong, K., Volpi, J. J., et al. (2023) · Cureus · doi:10.7759/cureus.37595 · Cited by 16
Friesen, K. B., Wu, L. Z., Waslen, A., & Lang, A. E. (2023) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2023.111596 · Cited by 16
Blickle, G., Böhm, F., & Wihler, A. (2023) · Personality and Individual Differences · doi:10.1016/j.paid.2023.112230 · Cited by 16
Koçak, B., Yardımcı, A. H., Nazlı, M. A., Yüzkan, S., Mutlu, S., Güzelbey, T., Özdemir, M. Ş., Akin, M., Yücel, S., Bulut, E., et al. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110893 · Cited by 16
Ständer, S., Fofana, F., Dias-Barbosa, C., Rodriguez, D. N., Budhiarso, I., Jabbar‐Lopez, Z. K., Piketty, C., Vernon, M., & Puelles, J. (2023) · Dermatology and Therapy · doi:10.1007/s13555-023-00962-8 · Cited by 16
Lee, T. T., Yang, D., Lai, K. K., et al. (2023) · JOR Spine · doi:10.1002/jsp2.1259 · Cited by 16
Tatakis, D. N., Paramitha, V., Lu, W., et al. (2023) · Journal of Periodontology · doi:10.1002/jper.23-0291 · Cited by 16
How long can naturalistic L2 pronunciation learning continue in adults? A 10-year study
Thomson, R. I., Derwing, T. M., & Munro, M. J. (2023) · Language Awareness · doi:10.1080/09658416.2023.2227559 · Cited by 19
Dogs’ expectations about occlusion events: from expectancy violation to exploration
Völter, C. J., Tomašić, A., Nipperdey, L., & Huber, L. (2023) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2023.0696 · Cited by 16
Smith, A. M., Mullaly, R., Franklin, O., et al. (2023) · Early Human Development · doi:10.1016/j.earlhumdev.2023.105832 · Cited by 16
Albaum, C. S., Sellitto, T., Vashi, N., Bohr, Y. M., & Weiss, J. A. (2023) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-023-06083-7 · Cited by 16
Adapting to a Robotic Era: The Transferability of Open and Laparoscopic Skills to Robotic Surgery
Farah, E., Abreu, A. A., Rail, B., et al. (2023) · Journal of surgical education · doi:10.1016/j.jsurg.2023.08.014 · Cited by 16
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
