The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 8,701–8,800 of 23,881
Ehmann, P., Beavan, A., Spielmann, J., Ruf, L., Mayer, J., Rohrmann, S., Nuß, C., & Englert, C. (2021) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2021.101952 · Cited by 34
Höhne, E., Recker, F., Schmok, E., Brossart, P., Raupach, T., & Schäfer, V. S. (2021) · Ultraschall in der Medizin - European Journal of Ultrasound · doi:10.1055/a-1528-1418 · Cited by 28
Kovačević, M., Klicov, L., Vukliš, D., Neblett, R., & Knežević, A. (2021) · Neurophysiologie Clinique · doi:10.1016/j.neucli.2021.06.005 · Cited by 32
Liu, Y., Xiao, B., Zhang, C., Li, J., Lai, Y., Shi, F., Shen, D. G., Wang, L., Sun, B., Li, Y., et al. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.731109 · Cited by 28
Rokonuzzaman, M., Kim, K., Dugar, K. K., & Fox, J. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2021.11.040 · Cited by 30
Age, Experience, and Business Performance: A Meta-Analysis of Work Unit-Level Effects
Guzzo, R. A., Nalbantian, H. R., & Anderson, N. (2021) · Work Aging and Retirement · doi:10.1093/workar/waab039 · Cited by 49
Real-life Wrist Movement Patterns Capture Motor Impairment in Individuals with Ataxia-Telangiectasia
Gupta, A. S., Luddy, A. C., Khan, N. C., Reiling, S., & Thornton, J. K. (2022) · The Cerebellum · doi:10.1007/s12311-022-01385-5 · Cited by 26
Kılınç, A. Ç., Polatcan, M., Turan, S., & Özdemir, N. (2022) · Irish Educational Studies · doi:10.1080/03323315.2022.2061567 · Cited by 42
Test-Retest Reliability of an Adaptive Thermal Pain Calibration Procedure in Healthy Volunteers
Amir, C., Rose-McCandlish, M., Weger, R., Dildine, T. C., Mischkowski, D., Necka, E. A., Lee, I., Wager, T. D., Pine, D. S., & Atlas, L. Y. (2022) · Journal of Pain · doi:10.1016/j.jpain.2022.01.011 · Cited by 28
Orosz, L. D., Bhatt, F. R., Jazini, E., et al. (2022) · Journal of Neurosurgery Spine · doi:10.3171/2022.5.spine22109 · Cited by 28
Chen, D., Hebert, M. A., & Wilson, J. (2022) · American Educational Research Journal · doi:10.3102/00028312221106773 · Cited by 28
Chen, Y., Farzan, R., Kraut, R. E., YeckehZaare, I., & Zhang, A. F. (2020) · Management Science · doi:10.1287/mnsc.2023.4852 · Cited by 30
Imaging Glaucomatous Damage Across the Temporal Raphe
Huang, G., Luo, T., Gast, T. J., et al. (2015) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.15-16730 · Cited by 28
Wetselaar, P., Koutris, M., Visscher, C. M., Larsson, P., John, M. T., & Lobbezoo, F. (2015) · Journal of Oral Rehabilitation · doi:10.1111/joor.12314 · Cited by 28
Reproducibility of velocity-dependent power: before and after lengthening contractions
Power, G. A., Dalton, B. H., Rice, C. L., & Vandervoort, A. A. (2011) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/h11-068 · Cited by 28
Skinfold Thickness at 8 Common Cryotherapy Sites in Various Athletic Populations
Jutte, L. S., Hawkins, J. R., Miller, K. C., et al. (2012) · Journal of Athletic Training · doi:10.4085/1062-6050-47.2.170 · Cited by 28
Psychometric properties of the Albanian version of the Orofacial Esthetic Scale: OES-ALB
Bimbashi, V., Čelebić, A., Staka, G., Asllani-Hoxha, F., Peršić, S., & Petričević, N. (2015) · BMC Oral Health · doi:10.1186/s12903-015-0083-x · Cited by 29
Acute seizure-suppressing effect of vagus nerve stimulation in the amygdala kindled rat
Rijkers, K., Aalbers, M. W., Hoogland, G., van Winden, L., Vles, J. S. H., Steinbusch, H. W. M., & Majoie, M. (2010) · Brain Research · doi:10.1016/j.brainres.2010.01.014 · Cited by 28
Performance and interaction anxiety: specific relationships with other- and self-evaluation concerns
Hook, J. N., Valentiner, D. P., & Connelly, J. (2012) · Anxiety Stress & Coping · doi:10.1080/10615806.2012.654777 · Cited by 28
Tucker, C. A., Montpetit, K., Bilodeau, N., et al. (2009) · Developmental Medicine & Child Neurology · doi:10.1111/j.1469-8749.2009.03267.x · Cited by 28
Salido‐Vallejo, R., Ruano, J., Garnacho-Saucedo, G. M., et al. (2014) · Clinical and Experimental Dermatology · doi:10.1111/ced.12398 · Cited by 28
Lord, S. P., Can, D., Yi, M., Marin, R., Dunn, C. W., Imel, Z. E., Georgiou, P., Narayanan, S. S., Steyvers, M., & Atkins, D. C. (2014) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2014.08.005 · Cited by 28
Validation of a buffet meal design in an experimental restaurant
Allirot, X., Saulais, L., Disse, E., Roth, H., Cazal, C., & Laville, M. (2012) · Appetite · doi:10.1016/j.appet.2012.02.011 · Cited by 28
Catastrophic Appraisal and Perceived Control as Moderators of Treatment Response in Panic Disorder
Meuret, A. E., Hofmann, S. G., & Rosenfield, D. (2010) · International Journal of Cognitive Therapy · doi:10.1521/ijct.2010.3.3.262 · Cited by 28
Estigarribia, B., Martin, G. E., & Roberts, J. E. (2012) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2012/10-0153 · Cited by 33
Individuality of the plasma sodium concentration
Zhang, Z., Duckart, J. P., Slatore, C. G., Fu, Y., Petrik, A. F., Thorp, M. L., & Cohen, D. (2014) · American Journal of Physiology-Renal Physiology · doi:10.1152/ajprenal.00585.2013 · Cited by 28
Weck, F., Rudari, V., Hilling, C., Hautzinger, M., Heidenreich, T., Schermelleh-Engel, K., & Stangier, U. (2013) · Psychiatry Research · doi:10.1016/j.psychres.2013.05.036 · Cited by 29
Manca, A., Pisanu, F., Ortu, E., de Natale, E. R., Ginatempo, F., Dragone, D., Tolu, E., & Deriu, F. (2015) · Gait & Posture · doi:10.1016/j.gaitpost.2015.04.005 · Cited by 28
The Bandim TBscore – reliability, further development, and evaluation of potential uses
Rudolf, F. (2014) · Global Health Action · doi:10.3402/gha.v7.24303 · Cited by 27
White, R., Cotton, S., Hind, J., Robbins, J., & Perry, A. R. (2008) · Dysphagia · doi:10.1007/s00455-008-9181-0 · Cited by 28
Framework for Consumer-Integrated Optimal Product Design
Smith, B. V., & Ierapepritou, M. G. (2009) · Industrial & Engineering Chemistry Research · doi:10.1021/ie900377e · Cited by 28
Mahling, M., Münch, A., Schenk, S., Volkert, S., Rein, A., Teichner, U., Piontek, P., Haffner, L., Heine, D., Manger, A., et al. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-185 · Cited by 27
Ditmyer, M. M., Dounis, G., Howard, K. M., et al. (2011) · BMC Oral Health · doi:10.1186/1472-6831-11-18 · Cited by 28
Assessment set for evaluation of clinical outcomes in multiple sclerosis: psychometric properties
Řasová, K., Martinková, P., Vyskotova, & Šedová (2012) · Patient Related Outcome Measures · doi:10.2147/prom.s32241 · Cited by 28
The Relation of Nasality and Nasalance to Nasal Port Area Based on a Computational Model
Bunton, K., & Story, B. H. (2011) · The Cleft Palate-Craniofacial Journal · doi:10.1597/11-131 · Cited by 28
Sources of patient uncertainty when reviewing medical disclosure and consent documentation
Donovan-Kicken, E., Mackert, M., Guinn, T. D., Tollison, A. C., & Breckinridge, B. (2012) · Patient Education and Counseling · doi:10.1016/j.pec.2012.10.007 · Cited by 28
Verbal fluency tests reliability in a Brazilian multicentric study, ELSA-Brasil
Passos, V. M. D. A., Giatti, L., Barreto, S. M., et al. (2011) · Arquivos de Neuro-Psiquiatria · doi:10.1590/s0004-282x2011000600017 · Cited by 28
LINKING O*NET DESCRIPTORS TO OCCUPATIONAL LITERACY REQUIREMENTS USING JOB COMPONENT VALIDATION
LaPolice, C. C., Carter, G. W., & JOHNSON, J. (2008) · Personnel Psychology · doi:10.1111/j.1744-6570.2008.00118.x · Cited by 28
Clinical utility of the oxygen uptake efficiency slope in cystic fibrosis patients
Gruet, M., Brisswalter, J., Mély, L., & Vallier, J. (2010) · Journal of Cystic Fibrosis · doi:10.1016/j.jcf.2010.03.003 · Cited by 27
An examination chair to measure internal rotation of the hip in routine settings: a validation study
Reichenbach, S., Jüni, P., Nüesch, E., Frey, F., Ganz, R., & Leunig, M. (2009) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2009.10.001 · Cited by 28
A generalizability theory analysis of group process ratings in the treatment of cocaine dependence
Crits-Christoph, P. F., Johnson, J. E., Gallop, R., Gibbons, M. B. C., Ring‐Kurtz, S., Hamilton, J. L., & Tu, X. M. (2011) · Psychotherapy Research · doi:10.1080/10503307.2010.551429 · Cited by 28
Keller, S. (2001) · American Journal of Industrial Medicine · doi:10.1002/ajim.10004 · Cited by 28
Eurelings-Bontekoe, E. H. M., Luyten, P., & Snellen, W. M. (2009) · Journal of Personality Assessment · doi:10.1080/00223890802634274 · Cited by 28
Ross, M. D. (2007) · Research in Sports Medicine · doi:10.1080/15438620701693298 · Cited by 28
Lilliengren, P., Falkenström, F., Sandell, R., Mothander, P. R., & Werbart, A. (2014) · Journal of Counseling Psychology · doi:10.1037/cou0000044 · Cited by 28
Reproducibility of Breast Arterial Calcium Mass Quantification Using Digital Mammography
Molloi, S. Y., Mehraien, T., Iribarren, C., et al. (2009) · Academic Radiology · doi:10.1016/j.acra.2008.08.011 · Cited by 28
Thinner cortex in the frontal lobes in mentally disordered offenders
Howner, K., Eskildsen, S. F., Fischer, H., Dierks, T., Wahlund, L., Jonsson, T., Wiberg, M. K., & Kristiansson, M. (2012) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2011.12.011 · Cited by 28
A structural model of the effects of preschool attention on kindergarten literacy
Dice, J. L., & Schwanenflugel, P. J. (2012) · Reading and Writing · doi:10.1007/s11145-011-9354-3 · Cited by 34
Salavati, M., Mazaheri, M., Khosrozadeh, F., et al. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9716-3 · Cited by 28
Effect of Heel Height on the Foot in Unilateral Standing.
Shimizu, M., & Andrew, P. D. (1999) · Journal of Physical Therapy Science · doi:10.1589/jpts.11.95 · Cited by 28
Development of a direct observation Measure of Environmental Qualities of Activity Settings
King, G. A., Rigby, P., Batorowicz, B., et al. (2014) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.12400 · Cited by 28
A comprehensive protocol for manual segmentation of the medial temporal lobe structures.
Moore, M., Hu, Y., Woo, S., O'Hearn, D., Iordan, A., Dolcos, S., & Dolcos, F. (2014) · Journal of Visualized Experiments · doi:10.3791/50991 · Cited by 43
Choi, B., Ko, S., Dobson, M., et al. (2014) · Ergonomics · doi:10.1080/00140139.2014.904008 · Cited by 28
Travis, L. A., Arthmire, S. J., Baig, A. M., Goldberg, A., & Malek, M. H. (2011) · Muscle & Nerve · doi:10.1002/mus.22211 · Cited by 28
Lee, J. N., Hsu, E. W., Rashkin, E., Thatcher, J. W., Kreitschitz, S., Gale, P. D., Healy, L., & Marchand, W. R. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.09.072 · Cited by 28
Speech sound articulation abilities of preschool-age children who stutter
Clark, C. E., Conture, E. G., Walden, T. A., & Lambert, W. E. (2013) · Journal of Fluency Disorders · doi:10.1016/j.jfludis.2013.09.004 · Cited by 28
Solbakken, O. A., Hansen, R. S., Havik, O., & Monsen, J. (2012) · Psychotherapy Research · doi:10.1080/10503307.2012.700871 · Cited by 32
González, M., Escobar, A., Herdman, M. J., Herrera-Espiñeira, C., García‐Pérez, L., & Sarasqueta, C. (2011) · Clinical Rheumatology · doi:10.1007/s10067-011-1855-6 · Cited by 28
Validated intraclass correlation statistics to test item performance models
Courrieu, P., Brand-D’abrescia, M., Peereman, R., Spieler, D. H., & Rey, A. (2010) · Behavior Research Methods · doi:10.3758/s13428-010-0020-5 · Cited by 28
Inter-rater reliability of a national acute stroke register
Varmdal, T., Ellekjær, H., Fjærtoft, H., Indredavik, B., Lydersen, S., & Bønaa, K. H. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1556-3 · Cited by 29
Tenhiälä, A., & Salvador, F. (2014) · Decision Sciences · doi:10.1111/deci.12076 · Cited by 28
Maskarinec, G., Morimoto, Y., Conroy, S. M., Pagano, I. S., & Franke, A. A. (2011) · The Journal of Nutrition · doi:10.3945/jn.110.133769 · Cited by 28
Eek, E., & Engardt, M. (2003) · Clinical Rehabilitation · doi:10.1177/026921550301700803 · Cited by 27
Chabot, G., Godin, G., & Gagnon, M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-93 · Cited by 28
The Effect of Pelvic Position on Popliteal Angle Achieved During 90:90 Hamstring-Length Test
Herrington, L. C. (2013) · Journal of Sport Rehabilitation · doi:10.1123/jsr.22.4.254 · Cited by 28
Competencies Through the Eyes of Psychologists: A closer look at assessing competencies
Heinsman, H., Hoogh, A. H. B. D., Koopman, P. L., & van Muijen, J. J. (2007) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2007.00400.x · Cited by 30
Neck flexor muscle fatigue in adolescents with headache – An electromyographic study
Oksanen, A., Pöyhönen, T., Metsähonkala, L., Anttila, P., Hiekkanen, H., Laimi, K., & Salminen, J. J. (2007) · European Journal of Pain · doi:10.1016/j.ejpain.2006.12.003 · Cited by 28
Gao, K. L., Chan, K.-M., Purves, S., & Tsang, W. W. N. (2014) · Journal of Orthopaedic Translation · doi:10.1016/j.jot.2014.07.003 · Cited by 28
Accuracy of chest radiographs in the emergency diagnosis of heart failure
Studler, U., Kretzschmar, M., Christ, M., Breidthardt, T., Noveanu, M., Schoetzau, A., Perruchoud, A. P., Steinbrich, W., & Mueller, C. E. (2008) · European Radiology · doi:10.1007/s00330-008-0930-0 · Cited by 28
Scapular dyskinesis in trapezius myalgia and intraexaminer reproducibility of clinical tests
Juul‐Kristensen, B., Hilt, K., Enoch, F., et al. (2011) · Physiotherapy Theory and Practice · doi:10.3109/09593985.2010.528548 · Cited by 28
Wang, F., He, Y., Guo, H., et al. (2010) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-09-0270 · Cited by 28
Arab, A. M., Rasouli, O., Amiri, M., & Tahan, N. (2013) · Chiropractic & Manual Therapies · doi:10.1186/2045-709x-21-37 · Cited by 28
Antituberculosis Drug Resistance in the South of Vietnam: Prevalence and Trends
Huong, N. T., Lan, N. T. N., Cobelens, F. G. J., et al. (2006) · The Journal of Infectious Diseases · doi:10.1086/507906 · Cited by 28
Porcerelli, J. H., Huth‐Bocks, A. C., Huprich, S. K., & Richardson, L. A. (2015) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2015.15020173 · Cited by 31
Davenport, T. E., Stevens, S. R., Baroni, K., Ness, J. M. V., & Snell, C. R. (2011) · Disability and Rehabilitation · doi:10.3109/09638288.2011.582925 · Cited by 28
Park, J., & Hopkins, J. T. (2012) · International Journal of Neuroscience · doi:10.3109/00207454.2012.725117 · Cited by 28
Peltonen, L., McCallum, L., Siirala, E., Haataja, M., Lundgrén‐Laine, H., Salanterä, S., & Lin, F. F. (2015) · BioMed Research International · doi:10.1155/2015/868653 · Cited by 28
Compton, M. T., Potts, A. A., Wan, C. R., et al. (2012) · Psychiatry Research · doi:10.1016/j.psychres.2012.07.041 · Cited by 28
McNeil, M., Buth, K. J., Brydie, A., et al. (2007) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/j.ejcts.2006.12.033 · Cited by 28
Sensitivity and Reliability of a Specific Test of Stroke Performance in Table Tennis
Mansec, Y. L., Dorel, S., Nordez, A., et al. (2015) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.2015-0444 · Cited by 28
Hamacher, D., Hamacher, D., Krowicki, M., & Schega, L. (2016) · Aging Clinical and Experimental Research · doi:10.1007/s40520-016-0536-3 · Cited by 27
Motealleh, A., Gheysari, E., Shokri, E., et al. (2016) · Manual Therapy · doi:10.1016/j.math.2016.02.002 · Cited by 28
Herold, I. H. F., Saporito, S., Bouwman, R. A., Houthuizen, P., van Assen, H. C., Mischi, M., & Korsten, H. H. M. (2015) · Cardiovascular Ultrasound · doi:10.1186/s12947-015-0044-1 · Cited by 28
Reproducibility of Resting State Connectivity in Patients with Stable Multiple Sclerosis
Pinter, D., Beckmann, C. F., Koini, M., Pirker, E., Filippini, N., Pichler, A., Fuchs, S., Fazekas, F., & Enzinger, C. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152158 · Cited by 30
Rossle, M. P., Blanke, P., Fritz, B., et al. (2016) · Journal of Vascular and Interventional Radiology · doi:10.1016/j.jvir.2016.03.028 · Cited by 28
Pelvic floor muscle reflex activity during coughing – an exploratory and reliability study
Luginbuehl, H., Baeyens, J., Kühn, A., et al. (2016) · Annals of Physical and Rehabilitation Medicine · doi:10.1016/j.rehab.2016.04.005 · Cited by 28
Longitudinal and Sex Measurement Invariance of the Affective Neuroscience Personality Scales
Orri, M., Rouquette, A., Pingault, J., Barry, C., Herba, C. M., Côté, S. M., & Berthoz, S. (2016) · Assessment · doi:10.1177/1073191116656795 · Cited by 29
Not all unprofessional behaviors are equal: The creation of a checklist of bad behaviors
Cullen, M. J., Konia, M. R., Borman-Shoap, E. C., Braman, J. P., Tiryaki, E., Marcus-Blank, B., & Andrews, J. S. (2016) · Medical Teacher · doi:10.1080/0142159x.2016.1231917 · Cited by 29
Atroszko, P. A., Bagińska, P., Mokosińska, M., & Atroszko, B. (2015) · CeON Repository (Centre for Evaluation in Education and Science) · Cited by 28
Exposure to Sodium Valproate during Pregnancy: Facial Features and Signs of Autism
Stadelmaier, R., Nasri, H. Z., Deutsch, C. K., Bauman, M. L., Hunt, A., Stodgell, C., Adams, J., & Holmes, L. B. (2017) · Birth Defects Research · doi:10.1002/bdr2.1052 · Cited by 28
Short- and long-term reliability of leg extensor power measurement in middle-aged and older adults
Hurst, C., Batterham, A. M., Weston, K. L., & Weston, M. (2017) · Journal of Sports Sciences · doi:10.1080/02640414.2017.1346820 · Cited by 31
Validity and Reliability of Visual Analog Scale Foot and Ankle: The Turkish Version
Gür, G., Turgut, E., Dilek, B., BALTACI, G. Y., Bek, N., & Yakut, Y. (2017) · The Journal of Foot & Ankle Surgery · doi:10.1053/j.jfas.2017.06.001 · Cited by 28
Jerant, A. F., Henderson, M. C., Griffin, E., Rainwater, J. A., Hall, T. R., Kelly, C., Peterson, E. M., Wofsy, D., & Franks, P. J. (2017) · BMC Medical Education · doi:10.1186/s12909-017-1030-0 · Cited by 30
The reliability and validity of the L-test in people with Parkinson’s disease
Haas, B. M., Clarke, E., Elver, L., Gowman, E., Mortimer, E., & Byrd, E. (2017) · Physiotherapy · doi:10.1016/j.physio.2017.11.218 · Cited by 29
Methods matter: A multi-trait multi-method analysis of student behavior
Miller, F. G., Johnson, A. H., Yu, H., Chafouleas, S. M., McCoach, D. B., Riley‐Tillman, T. C., Fabiano, G. A., & Welsh, M. E. (2018) · Journal of School Psychology · doi:10.1016/j.jsp.2018.01.002 · Cited by 28
Ozola‐Zālīte, I., Mark, E. B., Gudauskas, T., Lyadov, V. K., Olesen, S. S., Drewes, A. M., Puķītis, A., & Frøkjær, J. B. (2018) · European Journal of Clinical Nutrition · doi:10.1038/s41430-018-0110-5 · Cited by 28
Characterization of postural control impairment in women with fibromyalgia
Sempere-Rubio, N., López-Pascual, J., Aguilar-Rodríguez, M., Cortés-Amador, S., Espí-López, G., Villarrasa-Sapiña, I., & Serra-Añó, P. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196575 · Cited by 28
Bégel, V., Verga, L., Benoit, C.-É., Kotz, S. A., & Bella, S. D. (2018) · Annals of Physical and Rehabilitation Medicine · doi:10.1016/j.rehab.2018.04.001 · Cited by 31
Hu, B., Zhang, X., Mu, J., Wu, M. H., Zhu, Z., Liu, Z., & Wang, Y. (2018) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2018.2838767 · Cited by 28
Kim, S., Han, K., Seo, N., et al. (2018) · European Radiology · doi:10.1007/s00330-018-5520-1 · Cited by 28
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
