The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 9,901–10,000 of 23,881
The mechanical behavior of as received and retrieved nickel titanium orthodontic archwires
Lombardo, L., Toni, G., Mazzanti, V., et al. (2019) · Progress in Orthodontics · doi:10.1186/s40510-018-0251-z · Cited by 23
Carlson, J., Bellettiere, J., Kerr, J., Salmon, J., Timperio, A. F., Verswijveren, S. J. J. M., & Ridgers, N. D. (2019) · Journal of Sports Sciences · doi:10.1080/02640414.2019.1605646 · Cited by 24
Ferentinos, P. P., Yotsidi, V., Porichi, E., Douzenis, A., Papageorgiou, C., & Stalikas, A. (2019) · Journal of Clinical Psychology · doi:10.1002/jclp.22780 · Cited by 23
Brage, K., Hjarbæk, J., Kjær, P., Ingwersen, K. G., & Juul‐Kristensen, B. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-027725 · Cited by 23
Becker, N., Piccolo, L. D. R., & de Salles, J. F. (2019) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acz022 · Cited by 26
Jeong, A., Shin, D.-W., Park, J. H., & Park, K. (2019) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2019.05.020 · Cited by 23
Saldanha, G. S., Yarrow, J., Elsheikh, S. E., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.0621 · Cited by 23
Heslin, M., Chua, K., Trevillion, K., Nath, S., Howard, L. M., & Byford, S. (2019) · BJPsych Open · doi:10.1192/bjo.2019.71 · Cited by 23
Booth, C., Songco, A., Parsons, S. D., Heathcote, L. C., & Fox, E. (2019) · BMC Psychology · doi:10.1186/s40359-019-0342-8 · Cited by 26
Abraham, A., Drory, V. E., Fainmesser, Y., et al. (2019) · Muscle & Nerve · doi:10.1002/mus.26758 · Cited by 23
School and Classroom Climate Measures: Considerations for Use by State and Local Education Leaders
Schweig, J. D., Hamilton, L. S., & Baker, G. (2019) · RAND Corporation eBooks · doi:10.7249/rr4259 · Cited by 30
Reproducibility of flutter-range vibrotactile detection and discrimination thresholds
Mikkelsen, M. E., He, J. L., Tommerdahl, M., Edden, R. A. E., Mostofsky, S. H., & Puts, N. A. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-63208-z · Cited by 22
Daugherty, A. M., Schwarb, H. D., McGarry, M., Johnson, C. L., & Cohen, N. J. (2020) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01574 · Cited by 23
Romano, M., Moscovitch, D. A., Saini, P., & Huppert, J. D. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103651 · Cited by 28
Monfort-Pañego, M., & Miñana-Signes, V. (2020) · Measurement in Physical Education and Exercise Science · doi:10.1080/1091367x.2020.1784899 · Cited by 23
Donohue, C., Khalifa, Y., Perera, S., Sejdić, E., & Coyle, J. L. (2020) · Dysphagia · doi:10.1007/s00455-020-10191-2 · Cited by 28
Testing a convolutional neural network‐based hippocampal segmentation method in a stroke population
Zavaliangos‐Petropulu, A., Tubi, M. A., Haddad, E., Zhu, A. H., Braskie, M. N., Jahanshad, N., Thompson, P. M., & Liew, S. (2020) · Human Brain Mapping · doi:10.1002/hbm.25210 · Cited by 23
de Frutos-Valle, L., Martín, C., Alarcón, J. A., Palma-Fernández, J. C., Ortega, R., & Iglesias‐Linares, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-74488-w · Cited by 25
Hossein, A., Rabineau, J., Gorlier, D., Rio, J. I. J. D., van de Borne, P., Migeotte, P., & Nonclercq, A. (2021) · Sensors · doi:10.3390/s21030815 · Cited by 24
Smits-Engelsman, B. C. M., Smit, E., Doe-Asinyo, R. X., Lawerteh, S. E., Aertssen, W. F. M., Ferguson, G. D., & Jelsma, D. (2021) · BMC Pediatrics · doi:10.1186/s12887-021-02589-0 · Cited by 26
Ana, E. J. S., LaRowe, S. D., Gebregziabher, M., Morgan‐López, A. A., Lamb, K. E., Beavis, K. A., Bishu, K. G., & Martino, S. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.108716 · Cited by 23
Deep domain adaptation with ordinal regression for pain assessment using weakly-labeled videos
Praveen, R. G., Granger, É., & Cardinal, P. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104167 · Cited by 23
Perceived foreign accentedness: Acoustic distances and lexical properties
Porretta, V., Kyröläinen, A.-J., & Tucker, B. V. (2015) · Attention Perception & Psychophysics · doi:10.3758/s13414-015-0916-3 · Cited by 24
Analyzing cognitive presence in online courses using an artificial neural network
Harmon, S. W., & Mcklin, T. E. (2004) · Cited by 23
Bergmann, T. J., Sappok, T., Diefenbacher, A., & Dziobek, I. (2015) · Nordic Journal of Music Therapy · doi:10.1080/08098131.2015.1039567 · Cited by 23
Bergeron, L., Berthiaume, C., St‐Georges, M., Piché, G., & Smolla, N. (2013) · The Canadian Journal of Psychiatry · doi:10.1177/070674371305800805 · Cited by 23
Zinko, R. (2013) · Journal of Applied Social Psychology · doi:10.1111/jasp.12114 · Cited by 24
Repeatability of exhaled nitric oxide measurements in patients with COPD
Rouhos, A., Kainu, A., Piirilä, P. L., Sarna, S. J., Lindqvist, A. E., Karjalainen, J., & Sovijärvi, A. R. A. (2010) · Clinical Physiology and Functional Imaging · doi:10.1111/j.1475-097x.2010.00975.x · Cited by 23
Towards Automatic Evaluation of Learning Object Metadata Quality
Ochôa, X., & Duval, E. (2006) · Lecture notes in computer science · doi:10.1007/11908883_44 · Cited by 23
Kramer, U., de Roten, Y., Perry, J. C., & Despland, J. (2013) · The Journal of Nervous and Mental Disease · doi:10.1097/nmd.0b013e3182982982 · Cited by 23
Hasan, K. M., Moeller, F. G., & Narayana, P. A. (2014) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2014.01.014 · Cited by 27
A comparison of two scoring systems for endoscopic grading of gastric ulceration in horses
Bell, R., Kingston, J., & Mogg, T. D. (2007) · New Zealand Veterinary Journal · doi:10.1080/00480169.2007.36730 · Cited by 24
Rout, R., McDonnell, S., Benson, R., et al. (2010) · The Knee · doi:10.1016/j.knee.2010.04.010 · Cited by 23
Nishizono, A., Yamada, K., Khawplod, P., et al. (2012) · Vaccine · doi:10.1016/j.vaccine.2012.04.003 · Cited by 23
Lobular involution: localized phenomenon or field effect?
Vierkant, R. A., Hartmann, L. C., Pankratz, V. S., et al. (2008) · Breast Cancer Research and Treatment · doi:10.1007/s10549-008-0082-6 · Cited by 23
Zhang, C., Xu, Y., Cheng, Y., et al. (2015) · Archives of Oral Biology · doi:10.1016/j.archoralbio.2015.01.008 · Cited by 23
Rusby, J. C., Jones, L. B., Crowley, R. N., & Smolkowski, K. (2012) · Early Child Development and Care · doi:10.1080/03004430.2012.742992 · Cited by 23
Evaluation of an anti-rPA IgG ELISA for measuring the antibody response in mice
Little, S. F., Webster, W. M., NORRIS, S. L. W., et al. (2004) · Biologicals · doi:10.1016/j.biologicals.2004.02.001 · Cited by 23
Lyons, K. D., Tickle‐Degnen, L., Henry, A. D., et al. (2004) · Rehabilitation Psychology · doi:10.1037/0090-5550.49.4.328 · Cited by 23
Follow-up outcome in short-term group therapy for complicated grief.
Piper, W. E., Ogrodniczuk, J. S., Joyce, A. S., & Weideman, R. (2009) · Group Dynamics Theory Research and Practice · doi:10.1037/a0013830 · Cited by 24
Applicability of Latent Dirichlet Allocation to multi-disk search
Noel, G. E., & Peterson, G. L. (2014) · Digital Investigation · doi:10.1016/j.diin.2014.02.001 · Cited by 23
Muscle action and stress on the ribs in rowing
Wajswelner, H., Bennell, K. L., Story, I. H., et al. (2000) · Physical Therapy in Sport · doi:10.1054/ptsp.2000.0023 · Cited by 23
Weiler, P. J., & Bogoch, E. R. (1995) · The Journal Of Hand Surgery · doi:10.1016/s0363-5023(05)80140-6 · Cited by 23
Fick, L. J., Lang, E. V., Logan, H. L., Lutgendorf, S. K., & Benotsch, E. G. (1999) · Academic Radiology · doi:10.1016/s1076-6332(99)80164-0 · Cited by 23
A Reliability Study for Standing Functional Reach Test Using Modified and Traditional Rulers
Lin, Y.-H., Tang, Y.-W., Chen, T.-R., et al. (2012) · Perceptual and Motor Skills · doi:10.2466/15.03.10.pms.115.5.512-520 · Cited by 23
Keenan, G. M., Stocker, J., Barkauskas, V. H., Johnson, M. R., Maas, M., Moorhead, S. A., & Reed, D. (2003) · Journal of Nursing Measurement · doi:10.1891/jnum.11.2.135.57285 · Cited by 23
Finger enslaving in the dominant and non-dominant hand
Wilhelm, L., Martin, J. R., Latash, M. L., et al. (2013) · Human Movement Science · doi:10.1016/j.humov.2013.10.001 · Cited by 23
Clinical Measurement of Early Stuttering Severity: The Reliability of a 10-point Scale
Eve, C. L., Onslow, M., Andrews, C., et al. (1995) · Australian Journal of Human Communication Disorders · doi:10.3109/asl2.1995.23.issue-2.03 · Cited by 23
Rueter, M. A., Conger, R. D., & Ramisetty‐Mikler, S. (1999) · Family Relations · doi:10.2307/585684 · Cited by 23
A Model for the Assessment of Medical Students’ Competency in Medical Ethics
Favia, A., Frank, L. E., Gligorov, N., et al. (2013) · AJOB Primary Research · doi:10.1080/21507716.2013.768308 · Cited by 23
Mok, C., Zhou, L., McGrath, C. P. J., et al. (2007) · Acta Odontologica Scandinavica · doi:10.1080/00016350701771850 · Cited by 23
Twiss, C. O., MARTIN, S., Shore, R. E., et al. (2000) · The Journal of Urology · doi:10.1016/s0022-5347(05)67148-0 · Cited by 23
Gagnon, M., Légaré, F., Fortin, J., Lamothe, L., Labrecque, M., & Duplantie, J. (2008) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-8-17 · Cited by 24
Fine temporal structure of cardiorespiratory synchronization
Ahn, S., Solfest, J., & Rubchinsky, L. L. (2013) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00314.2013 · Cited by 23
Agarwal, S., Allison, G. T., & Singer, K. P. (2005) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2005.07.005 · Cited by 23
Vatne, T. M., Ruland, C. M., Ørnes, K., et al. (2011) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsr074 · Cited by 23
Adriaansen, M. J. M., van Achterberg, T., & Borm, G. F. (2008) · Journal of Professional Nursing · doi:10.1016/j.profnurs.2007.10.003 · Cited by 23
Inter-rater reliability of the Early Functional Abilities scale
Alvsåker, K., Walther, S., Kleffelgård, I., et al. (2011) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-0855 · Cited by 23
Assessing Trainee Clinical Psychologists' Clinical Competence
Tweed, A. E., Graber, R., & Wang, M. (2010) · Psychology Learning & Teaching · doi:10.2304/plat.2010.9.2.50 · Cited by 24
Kienbacher, T., Habenicht, R., Starek, C., et al. (2014) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-11-106 · Cited by 23
Luterek, J. A., Orsillo, S. M., & Marx, B. P. (2005) · Journal of Traumatic Stress · doi:10.1002/jts.20027 · Cited by 23
Hoyte, L. P. J., Brubaker, L. T., Fielding, J. R., Lockhart, M. E., Heilbrun, M. E., Salomon, C. G., Ye, W., & Brown, M. B. (2009) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21847 · Cited by 23
Evaluation of two self-referent foot health instruments
Trevethan, R. (2010) · The Foot · doi:10.1016/j.foot.2010.07.001 · Cited by 23
Nyland, J. A., Burden, R., Krupp, R. J., et al. (2011) · Clinical Biomechanics · doi:10.1016/j.clinbiomech.2010.11.021 · Cited by 23
Gude, T., Grimstad, H., Holen, A., et al. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0508-x · Cited by 23
Hamacher, D., Hamacher, D., Törpel, A., Krowicki, M., Herold, F., & Schega, L. (2015) · Gait & Posture · doi:10.1016/j.gaitpost.2015.12.021 · Cited by 23
Ayupe, K. M. A., de Albuquerque, K. A., Ferreira, M. L. A., et al. (2016) · Brazilian Journal of Physical Therapy · doi:10.1590/bjpt-rbf.2014.0131 · Cited by 23
Social responses to Facebook status updates: The role of extraversion and social anxiety
Deters, F. G., Mehl, M. R., & Eid, M. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.02.093 · Cited by 30
Behavioral activation for major depression in adolescents: Results from a pilot study.
Ritschel, L. A., Ramirez, C. L., Cooley, J. L., & Craighead, W. E. (2016) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12140 · Cited by 23
Macnab, A. J., Levine, M., Glick, N., Phillips, N., Susak, L. E., & Elliott, M. (1994) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/bf03010934 · Cited by 22
Special Feature: The Home Assessment Profile—A Reliable and Valid Assessment Tool
Chandler, J. M., Duncan, P. W., Weiner, D. K., et al. (2001) · Topics in Geriatric Rehabilitation · doi:10.1097/00013614-200103000-00010 · Cited by 23
Yamamoto, Y. Y., Yamaguchi, S., Sasho, T., et al. (2017) · Radiology · doi:10.1148/radiol.2016160695 · Cited by 23
A TEST OF THE CIRCUMPLEX MODEL OF MARITAL AND FAMILY SYSTEMS USING THE CLINICAL RATING SCALE
Thomas, V. K., & Ozechowski, T. J. (2000) · Journal of Marital and Family Therapy · doi:10.1111/j.1752-0606.2000.tb00321.x · Cited by 23
Evaluation of the reliability of reflective marker placements
Marks, R., & Karkouti, E. (1996) · Physiotherapy Research International · doi:10.1002/pri.47 · Cited by 23
Test-Retest Reliability of an Abbreviated Self-Report Overall Health Status Measure
Hart, D. L. (2003) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2003.33.12.734 · Cited by 23
Straight leg raise test high reliability is not a motor memory artefact
Chow, R., Adams, R. D., & Herbert, R. (1994) · Australian Journal of Physiotherapy · doi:10.1016/s0004-9514(14)60457-8 · Cited by 23
Rorschach Personality Predictors of Weight Loss With Behavior Modification in Obesity Treatment
Elfhag, K., Rössner, S., Lindgren, T. W., Andersson, I., & Carlsson, A. M. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8303_11 · Cited by 23
Cicchetti, D. V., & Heavens, R. (1981) · Educational and Psychological Measurement · doi:10.1177/001316448104100119 · Cited by 23
Repeatability of power Doppler sonography of cervical lymph nodes
Ying, M. T. C., Ahuja, A. T., & Brook, F. M. (2002) · Ultrasound in Medicine & Biology · doi:10.1016/s0301-5629(02)00523-9 · Cited by 23
Brouwer, B. J., Mazzoni, C. F., & Pearce, G. W. (2001) · Ergonomics · doi:10.1080/00140130010017868 · Cited by 23
Physical Attractiveness, Need for Approval, Social Self-Esteem, and Maladjustment
O’Grady, K. E. (1989) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.1989.8.1.62 · Cited by 23
Anderson, N. R., & Öneş, D. S. (2003) · European Journal of Personality · doi:10.1002/per.484 · Cited by 23
Inter‐ and intra‐tester reliability of the balance performance monitor in a non‐patient population
Haas, B. M., & Whitmarsh, T. E. (1998) · Physiotherapy Research International · doi:10.1002/pri.132 · Cited by 23
A tool for measuring the asthma self-management competency of families
Taylor, G. H., Rea, H. H., McNaughton, S., et al. (1991) · Journal of Psychosomatic Research · doi:10.1016/0022-3999(91)90043-n · Cited by 23
High frequency eye tremor: reliability of measurement
Bolger, C., Sheahan, N. F., Coakley, D., & Malone, J. F. (1992) · Clinical Physics and Physiological Measurement · doi:10.1088/0143-0815/13/2/007 · Cited by 24
An assessment of structured diagnostic interviews for adult anxiety disorders
Page, A. C. (1991) · International Review of Psychiatry · doi:10.3109/09540269109110406 · Cited by 23
Bridgeford, E., Wang, S., Yang, Z., Wang, Z., Xu, T., Craddock, C., Dey, J. K., Kiar, G., Gray-Roncal, W., Colantuoni, C., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/802629 · Cited by 22
Goran, M. I., & Khaled, M. A. (1995) · Obesity Research · doi:10.1002/j.1550-8528.1995.tb00187.x · Cited by 22
The influence of occupation on lumbar sagittal motion and posture
Milosavljevic, S., Milburn, P. D., & Knox, B. W. (2005) · Ergonomics · doi:10.1080/00140130500070848 · Cited by 22
Speech-Language Dissociations, Distractibility, and Childhood Stuttering
Clark, C. E., Conture, E. G., Walden, T. A., & Lambert, W. E. (2015) · American Journal of Speech-Language Pathology · doi:10.1044/2015_ajslp-14-0198 · Cited by 22
Interrater reliability for multilevel data: A generalizability theory approach.
ten Hove, D., Jorgensen, T. D., & van der Ark, L. A. (2022) · Psychological Methods · doi:10.1037/met0000391 · Cited by 22
Exploring Differences in Measurement and Reporting of Classroom Observation Inter-Rater Reliability
Wilhelm, A. G., Rouse, A. G., & Jones, F. (2018) · University of Massachusetts Amherst · doi:10.7275/at67-md25 · Cited by 20
Student perceptions of stylistic variation in introductory university textbooks
Egbert, J. (2013) · Linguistics and Education · doi:10.1016/j.linged.2013.09.007 · Cited by 22
“Can you read my mind?” Conventionalized indirect requests and Theory of Mind abilities
Marocchini, E., & Domaneschi, F. (2022) · Journal of Pragmatics · doi:10.1016/j.pragma.2022.03.011 · Cited by 22
A transdiagnostic prodrome for severe mental disorders: an electronic health record study
Arribas, M., Oliver, D., Patel, R., Kornblum, D., Shetty, H., Damiani, S., Krakowski, K., Provenzani, U., Stahl, D. R., Koutsouleris, N., et al. (2024) · Molecular Psychiatry · doi:10.1038/s41380-024-02533-5 · Cited by 22
Gomez, R., Zarate, D., Brown, T., Hein, K., & Stavropoulos, V. (2024) · Clinical Psychologist · doi:10.1080/13284207.2024.2341816 · Cited by 26
García-Herreros, S., Gómez, J. J. L., Cebriá, Á., Izaola, O., Salvador-Coloma, P., Nozal, S., Cano, J., Primo, D., Godoy, E. J., & de Luis, D. A. (2024) · Nutrients · doi:10.3390/nu16121806 · Cited by 22
Comparison of Three Commercially Available, AI-Driven Cephalometric Analysis Tools in Orthodontics
Kazimierczak, W., Gawin, G., Janiszewska‐Olszowska, J., et al. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13133733 · Cited by 22
Mayorga-Vega, D., & Viciana, J. (2015) · PubMed · doi:10.3305/nh.2015.32.1.8919 · Cited by 22
Park, B.-S., Kim, M.-Y., Lee, L.-K., Yang, S., Lee, W.-D., Noh, J.-W., Shin, Y.-S., Kim, J., Lee, J.-U., Kwak, T.-Y., et al. (2015) · Journal of Physical Therapy Science · doi:10.1589/jpts.27.1603 · Cited by 22
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
