The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 5,101–5,200 of 23,883
Chuang, L.-L., Lin, K., Wu, C., Chang, C., Chen, H., Yin, H.-P., & Wang, L. (2012) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2012.08.212 · Cited by 56
Advancing the case for creativity through graduate business education
Dewett, T. C., & Gruys, M. L. (2007) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2007.04.001 · Cited by 56
Coordination dynamics in horse-rider dyads
Wolframm, I., Bosga, J., & Meulenbroek, R. G. J. (2013) · Human Movement Science · doi:10.1016/j.humov.2012.11.002 · Cited by 65
Bender, A. R., & Raz, N. (2012) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2011.12.025 · Cited by 56
Callaghan, G. M., Follette, W. C., Ruckstuhl, L. E., & Linnerooth, P. J. N. (2008) · The Behavior Analyst Today · doi:10.1037/h0100648 · Cited by 55
Development of the Therapist Empathy Scale
Decker, S. E., Nich, C., Carroll, K. M., & Martino, S. (2013) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465813000039 · Cited by 63
Gallezot, J., Zheng, M., Lim, K., Lin, S.-F., Labaree, D. C., Matuskey, D., Huang, Y. H., Ding, Y., Carson, R. E., & Malison, R. T. (2014) · Journal of Nuclear Medicine · doi:10.2967/jnumed.113.132928 · Cited by 56
Alizai, H., Virayavanich, W., Joseph, G. B., et al. (2014) · Radiology · doi:10.1148/radiol.13122056 · Cited by 56
Topuz, S., Ülger, Ö., Yakut, Y., & Sener, F. (2011) · Prosthetics and Orthotics International · doi:10.1177/0309364611407678 · Cited by 56
Heppt, B., Haag, N., Böhme, K., & Stanat, P. (2014) · Reading Research Quarterly · doi:10.1002/rrq.83 · Cited by 56
Barber, J., Gallop, R., Crits-Christoph, P., Barrett, M. S., Klostermann, S., McCarthy, K., & Sharpless, B. A. (2008) · Psychoanalytic Psychology · doi:10.1037/0736-9735.25.3.461 · Cited by 58
Immediate effects of spinal manipulation on nitric oxide, substance P and pain perception
Molina, F., Lomas‐Vega, R., Hita‐Contreras, F., et al. (2014) · Manual Therapy · doi:10.1016/j.math.2014.02.007 · Cited by 56
3D liver segmentation using multiple region appearances and graph cuts
Peng, J., Hu, P., Lu, F., Peng, Z., Kong, D.-X., & Zhang, H. (2015) · Medical Physics · doi:10.1118/1.4934834 · Cited by 56
Cui, Y., Song, J., Pollom, E. L., Alagappan, M., Shirato, H., Chang, D. T., Koong, A. C., & Li, R. (2016) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2016.04.034 · Cited by 56
A novel smartphone ophthalmic imaging adapter: User feasibility studies in Hyderabad, India
Chang, R., Ludwig, C. A., Murthy, S. I., et al. (2016) · Indian Journal of Ophthalmology · doi:10.4103/0301-4738.181742 · Cited by 56
Temporal Evolution of Brain Functional Connectivity Metrics: Could 7 Min of Rest be Enough?
Tomasi, D. G., Kojori, E. S., & Volkow, N. D. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw227 · Cited by 59
Ouyang, M., Liu, P., Jeon, T., Chalak, L. F., Heyne, R. J., Rollins, N. K., Licht, D. J., Detre, J. A., Roberts, T. P. L., Lu, H., et al. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.12.034 · Cited by 57
Describe yourself to improve your autobiographical memory: A study in Alzheimer's disease
Haj, M. E., & Antoine, P. (2017) · Cortex · doi:10.1016/j.cortex.2017.01.004 · Cited by 58
Exploration and Firm Performance: The Moderating Impact of Competitive Strategy
Su, Z., Guo, H., & Sun, W. (2017) · British Journal of Management · doi:10.1111/1467-8551.12218 · Cited by 56
Clemens, S. M., GAILEY, R. S., Bennett, C., Pasquina, P. F., Kirk-Sanchez, N. J., & GAUNAURD, I. A. (2017) · Clinical Rehabilitation · doi:10.1177/0269215517728324 · Cited by 56
Devocht, E., Janssen, A. M., Chalupper, J., Stokroos, R., & George, E. (2017) · Trends in Hearing · doi:10.1177/2331216517727900 · Cited by 55
Busch, T., Sangen, A., Vanpoucke, F. J., & Wieringen, A. V. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0960-0 · Cited by 55
Umbach, R., Raine, A., & Leonard, N. R. (2017) · Criminal Justice and Behavior · doi:10.1177/0093854817736345 · Cited by 62
Stolaki, A., & Economides, A. A. (2018) · Computers & Education · doi:10.1016/j.compedu.2018.05.009 · Cited by 66
Liepertz, S., & Borowski, A. (2018) · International Journal of Science Education · doi:10.1080/09500693.2018.1478165 · Cited by 49
Yeung, L., Olsen, R. K., Hong, B., Mihajlovic, V., D’Angelo, M. C., Kacollja, A., Ryan, J. D., & Barense, M. D. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01385 · Cited by 60
Reliability and validity analyzes of Kinect V2 based measurement system for shoulder motions
Çubukçu, B., Yüzgeç, U., Zı̇lelı̇, R., & Zı̇lelı̇, A. (2019) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2019.10.017 · Cited by 56
Brimhall, K. C., Fenwick, K. M., Farahnak, L. R., Hurlburt, M. S., Roesch, S. C., & Aarons, G. A. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0670-9 · Cited by 59
van de Bunt, F., Pearl, M. L., Lee, E. K. P., et al. (2015) · Skeletal Radiology · doi:10.1007/s00256-015-2207-4 · Cited by 56
Hayas, C. L., Quintana, J. M., Padierna, Á., Bilbao, A., Muñoz, P., Madrazo, A., Urresti, B., & Cook, E. F. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.06.005 · Cited by 56
The effects of cognitive complexity on a social sequencing task in schizophrenia
Corrigan, P. W., & Addis, I. B. (1995) · Schizophrenia Research · doi:10.1016/0920-9964(94)00072-g · Cited by 56
Relative effectiveness and validity of mood induction procedures: a meta-analysis
Westermann, R., Spies, K., Stahl, G., & Hesse, F. W. (1996) · European Journal of Social Psychology · doi:10.1002/(sici)1099-0992(199607)26:4 · Cited by 1,134
Higgins, J., Bezjak, A., Franks, K., et al. (2008) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2008.06.1926 · Cited by 56
Variability of grip strength during isometric contraction
Fairfax, A. H., Balnave, R. J., & Adams, R. D. (1995) · Ergonomics · doi:10.1080/00140139508925229 · Cited by 56
Proteinuria in obstructive sleep apnea
Casserly, L. F., Chow, N., Ali, S., Gottlieb, D. J., Epstein, L. J., & Kaufman, J. S. (2001) · Kidney International · doi:10.1046/j.1523-1755.2001.00952.x · Cited by 55
The relation between worrying and concerns: The importance of perceived probability and cost
Berenbaum, H., Thompson, R. J., & Pomerantz, E. M. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2006.03.009 · Cited by 55
Dillen, L. R. V., McDonnell, M. K., Fleming, D. A., et al. (2000) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2000.30.6.307 · Cited by 56
Knee extensor torque and quadriceps femoris EMG during perceptually-guided isometric contractions
Pincivero, D. M., Coelho, A. J., Campy, R. M., et al. (2003) · Journal of Electromyography and Kinesiology · doi:10.1016/s1050-6411(02)00096-2 · Cited by 56
Measurements of the perceived ward milieu: a reevaluation of the Ward Atmosphere Scale
Friis, S. (1986) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1986.tb02729.x · Cited by 56
A Comparison of Four Methods of Obtaining a Negative Impression of the Foot
Laughton, C. A., DAVIS, I. M., & Williams, D. S. B. (2002) · Journal of the American Podiatric Medical Association · doi:10.7547/87507315-92-5-261 · Cited by 56
Accuracy of self-assessment of the minimal record of disability in patients with multiple sclerosis
Solari, A., Amato, M. P., Bergamaschi, R., Logroscino, G., Citterio, A., Bochicchio, D., & Filippini, G. (2009) · Acta Neurologica Scandinavica · doi:10.1111/j.1600-0404.1993.tb04073.x · Cited by 56
Maninger, N., Capitanio, J. P., Mendoza, S. P., & Mason, W. A. (2003) · American Journal of Primatology · doi:10.1002/ajp.10111 · Cited by 56
Bazzano, L. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf003 · Cited by 56
Measurement of Isometric Force in Children with and without Duchenne's Muscular Dystrophy
Brussock, C. M., Haley, S. M., Munsat, T. L., et al. (1992) · Physical Therapy · doi:10.1093/ptj/72.2.105 · Cited by 56
Alignment of an Alternate Assessment with State Academic Standards
Roach, A. T., Elliott, S. N., & Webb, N. L. (2005) · The Journal of Special Education · doi:10.1177/00224669050380040301 · Cited by 56
Factorial Validity of Student Ratings of Instruction
Renaud, R. D., & Murray, H. G. (2005) · Research in Higher Education · doi:10.1007/s11162-005-6934-6 · Cited by 56
Laughing at the Looking Glass: Does Humor Style Serve as an Interpersonal Signal?
Zeigler‐Hill, V., Avi Besser, A., & Jett, S. E. (2013) · Evolutionary Psychology · doi:10.1177/147470491301100118 · Cited by 64
A prototype strategy for construction of personality scales.
Broughton, R. (1984) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.47.6.1334 · Cited by 56
Effects of television violence on memory for commercial messages.
Bushman, B. J. (1998) · Journal of Experimental Psychology Applied · doi:10.1037/1076-898x.4.4.291 · Cited by 56
Simmenroth-Nayda, A., Heinemann, S., Nolte, C., Fischer, T., & Himmel, W. (2014) · International Journal of Medical Education · doi:10.5116/ijme.5454.c665 · Cited by 54
Lousada, M., Jesus, L. M. T., Hall, A., & Joffe, V. (2014) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12095 · Cited by 54
Examining self-presentation as a motivational explanation for comparative optimism.
Tyler, J. M., & Rosier, J. G. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0016231 · Cited by 54
Toys that squeak: Toy type impacts quality and quantity of parent–child interactions
Miller, J., Lossia, A. K., Suárez-Rivera, C., & Gros-Louis, J. J. (2017) · First Language · doi:10.1177/0142723717714947 · Cited by 55
Hedge, C., Vivian-Griffiths, S., Powell, G., Bompas, A., & Sumner, P. (2019) · Consciousness and Cognition · doi:10.1016/j.concog.2019.102797 · Cited by 60
The Progressive Supranuclear Palsy Clinical Deficits Scale
Piot, I., Schweyer, K., Respondek, G., Stamelou, M., DescribePSP study group, ProPSP study group, MDS‐endorsed PSP study group, Sckopke, P., Schenk, T., Goetz, C. G., et al. (2020) · Movement Disorders · doi:10.1002/mds.27964 · Cited by 57
Hoemann, K., Barrett, L. F., & Quigley, K. S. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.704125 · Cited by 66
Wilson, C. A., Metwally, H., Heavner, S. F., Kennedy, A. B., & Britt, T. W. (2021) · International Journal of Mental Health Nursing · doi:10.1111/inm.12942 · Cited by 59
Reliability and validity of the 6-min walk test in adults and seniors with intellectual disabilities
Guerra-Balic, M., Oviedo, G. R., Javierre, C. F., Fortuño, J., Barnet-López, S., Méndez, O. A. N., Álamo, J., & Fernhall, B. (2015) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2015.09.011 · Cited by 56
McCool-Myers, M., Wahl, J., Schlecht, I., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0139895 · Cited by 55
Semiautomatic Volumetric Tumor Segmentation for Hepatocellular Carcinoma
Tacher, V., Lin, M., Chao, M., Gjesteby, L., Bhagat, N., Mahammedi, A., Ardon, R., Mory, B., & Geschwind, J.-F. H. (2013) · Academic Radiology · doi:10.1016/j.acra.2012.11.009 · Cited by 54
Motivation as a Mediator of Social Disparities in Academic Achievement
Steinmayr, R., Dinger, F. C., & Spinath, B. (2011) · European Journal of Personality · doi:10.1002/per.842 · Cited by 76
Prevalence, knowledge, and treatment of transient ischemic attacks in China
Wang, Y., Zhao, X., Jiang, Y., Li, H., Wang, L., Johnston, S. C., Liu, L., Wong, K. S., Wang, C., Pan, Y., et al. (2015) · Neurology · doi:10.1212/wnl.0000000000001665 · Cited by 55
Fulbrook, P., & Anderson, A. (2015) · Journal of Advanced Nursing · doi:10.1111/jan.12825 · Cited by 62
Mobini, A., Behzadipour, S., & Saadat, M. (2015) · BioMedical Engineering OnLine · doi:10.1186/s12938-015-0070-0 · Cited by 54
The prevalence and characteristics of relational depth events in psychotherapy
Wiggins, S., Elliott, R., & Cooper, M. (2011) · Psychotherapy Research · doi:10.1080/10503307.2011.629635 · Cited by 55
Aridon, P., Solari, A., Motta, A., Mendozzi, L., Bergamaschi, R., Ghezzi, A., Mancardi, G., Milanese, C., Montanari, E., & Pucci, E. (2004) · Neurological Sciences · doi:10.1007/s10072-003-0192-1 · Cited by 55
Furlan, J. C., Fehlings, M. G., Massicotte, E. M., Aarabi, B., Vaccaro, A. R., Bono, C. M., Madrazo, I., Villanueva, C., Grauer, J. N., & Mikulis, D. J. (2007) · Spine · doi:10.1097/brs.0b013e318145a91c · Cited by 55
Psychometric properties of the Contingency Management Competence Scale☆
Petry, N. M., Alessi, S. M., Ledgerwood, D. M., & Sierra, S. (2010) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2009.12.027 · Cited by 55
Mouri, N., Sarkar, M., & Frye, M. B. (2011) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2011.02.002 · Cited by 55
Inter- and intrarater reliability of the Waterlow pressure sore risk scale: A systematic review
Kottner, J., Dassen, T. W. N., & Tannen, A. (2008) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2008.09.010 · Cited by 56
Pryseley, A., Mintiens, K., Knapen, K., der Stede, Y. V., & Molenberghs, G. (2010) · Journal of Applied Statistics · doi:10.1080/02664760903150706 · Cited by 55
Four Square Step Test in ambulant persons with multiple sclerosis
Wagner, J. M., Norris, R. A., Dillen, L. R. V., Thomas, F. P., & Naismith, R. T. (2013) · International Journal of Rehabilitation Research · doi:10.1097/mrr.0b013e32835fd97f · Cited by 56
Reliability of classification systems for subaxial cervical injuries
Stone, A., Bransford, R. J., Lee, M. J., et al. (2010) · Evidence-Based Spine-Care Journal · doi:10.1055/s-0030-1267064 · Cited by 55
A comparison of community‐dwelling older schizophrenia patients by residential status
Auslander, L. A., Lindamer, L. A., DeLaPena, J., Harless, K., Polichar, D., Patterson, T. L., Zisook, S., & Jeste, D. V. (2001) · Acta Psychiatrica Scandinavica · doi:10.1034/j.1600-0447.2001.00262.x · Cited by 55
Axis V – Global Assessment of Functioning Scale (GAF), further evaluation of the self-report version
Ramirez, A., Ekselius, L., & Ramklint, M. (2008) · European Psychiatry · doi:10.1016/j.eurpsy.2008.05.001 · Cited by 55
Haeseler, F., Fortin, A. H., Pfeiffer, C. A., Walters, C. A., & Martino, S. (2010) · Patient Education and Counseling · doi:10.1016/j.pec.2010.10.029 · Cited by 55
Feasibility of online divergent thinking assessment
Hass, R. W. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2014.12.056 · Cited by 64
Can direct manipulation lower the barriers to computer programming and promote transfer of training?
Hundhausen, C. D., Farley, S., & Brown, J. L. (2009) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1592440.1592442 · Cited by 59
Empirical Evidence for the Nature of Holistic Comfort
Kolcaba, K. Y., & Steiner, R. P. (2000) · Journal of Holistic Nursing · doi:10.1177/089801010001800106 · Cited by 55
Lindahl, K. M., Malik, N., Kaczynski, K. J., & Simons, J. (2004) · Development and Psychopathology · doi:10.1017/s0954579404004699 · Cited by 59
Manson, J. H., & Perry, S. (2013) · Journal of comparative psychology · doi:10.1037/a0031316 · Cited by 55
Norms and comprehension times for 305 French idiomatic expressions
Bonin, P., Méot, A., & Bugaïska, A. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0331-4 · Cited by 55
A METHODandRESOURCESforASSESSINGtheReliabilityofSimulation Evaluation Instruments
Adamson, K. A., & Kardong‐Edgren, S. (2012) · Nursing Education Perspectives · doi:10.5480/1536-5026-33.5.334 · Cited by 55
Fingleton, C., Dempsey, L., Smart, K. M., & Doody, C. (2014) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2013.12.006 · Cited by 54
Mylonas, I., Kazmi, M. H., Fuller, L., deKemp, R. A., Yam, Y., Chen, L., Beanlands, R. S., & Chow, B. J. W. (2012) · European Heart Journal - Cardiovascular Imaging · doi:10.1093/ehjci/jes079 · Cited by 54
Huber, G. P., Miller, C. C., & Glick, W. H. (1990) · Organization Science · doi:10.1287/orsc.1.1.11 · Cited by 55
Myers, A. M., Trang, A., & Crizzle, A. M. (2011) · Canadian Journal on Aging / La Revue canadienne du vieillissement · doi:10.1017/s0714980811000481 · Cited by 55
Method for Cortical Bone Structural Analysis From Magnetic Resonance Images1
Gomberg, B. R., Saha, P. K., & Wehrli, F. W. (2005) · Academic Radiology · doi:10.1016/j.acra.2005.06.012 · Cited by 55
Enablers of top management team support for integrated management control systems innovations
Lee, J., Elbashir, M. Z., Mahama, H., & Sutton, S. G. (2013) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2013.07.001 · Cited by 54
Pereira, L. M., Dias, J. M., Mazuquin, B., Castanhas, L. G., Menacho, M. D. O., & Cardoso, J. R. (2013) · Brazilian Journal of Physical Therapy · doi:10.1590/s1413-35552012005000091 · Cited by 55
Acoustic and Articulatory Features of Diphthong Production: A Speech Clarity Study
Tasko, S. M., & Greilick, K. L. (2009) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2009/08-0124 · Cited by 55
How Are Personality Traits Related to Preparation for Future Care Needs in Older Adults?
Sörensen, S., Duberstein, P. R., Chapman, B. P., Lyness, J. M., & Pinquart, M. (2008) · The Journals of Gerontology Series B · doi:10.1093/geronb/63.6.p328 · Cited by 55
Students' Perception of School and Their Smoking and Alcohol Use: A Cross-National Study
Samdal, O., Wold, B., Klepf, K. I., & Kannas, L. K. (2000) · Addiction Research & Theory · doi:10.3109/16066350009004417 · Cited by 55
Iconicity and the Emergence of Combinatorial Structure in Language
Verhoef, T., Kirby, S., & de Boer, B. (2015) · Cognitive Science · doi:10.1111/cogs.12326 · Cited by 56
Greene, B. R., Foran, T. G., McGrath, D., et al. (2012) · Journal of Applied Biomechanics · doi:10.1123/jab.28.3.349 · Cited by 55
Richmond, S. A., Kang, J., Doyle–Baker, P. K., Nettel‐Aguirre, A., & Emery, C. A. (2015) · Clinical Journal of Sport Medicine · doi:10.1097/jsm.0000000000000261 · Cited by 55
Schmitz, F., Schnabel, K., Stricker, D., Fischer, M. R., & Guttormsen, S. (2017) · Patient Education and Counseling · doi:10.1016/j.pec.2017.01.016 · Cited by 55
Fouche, P. F., Stein, C. O. A., Simpson, P., Carlson, J. N., & Doi, S. A. R. (2017) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.03.026 · Cited by 55
The Essential Properties of Yoga Questionnaire (EPYQ): Psychometric Properties
Park, C. L., Elwy, A. R., Maiya, M., Sarkin, A. J., Riley, K. E., Eisen, S. V., Gutierrez, I. A., Finkelstein‐Fox, L., Lee, S. Y., Casteel, D., et al. (2018) · International Journal of Yoga Therapy · doi:10.17761/2018-00016r2 · Cited by 63
Kyrölä, K., Järvenpää, S. K., Tuija, J., et al. (2018) · Neurospine · doi:10.14245/ns.1836054.027 · Cited by 55
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
