The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 9,201–9,300 of 23,881
McLeod, B. D., Southam‐Gerow, M. A., & Kendall, P. C. (2017) · Psychological Assessment · doi:10.1037/pas0000465 · Cited by 26
Hesper, T., Bixby, S. D., Kim, Y. J., et al. (2017) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.16.01262 · Cited by 26
Systematic Review of In-Session Affect Experience in Cognitive Behavioral Therapy for Depression
Doorn, K. A., & Barber, J. P. (2017) · Cognitive Therapy and Research · doi:10.1007/s10608-017-9865-6 · Cited by 27
Santos, C., Pauchard, N., & Guilloteau, A. (2017) · Hand surgery & rehabilitation · doi:10.1016/j.hansur.2017.06.005 · Cited by 26
Tey, W. K., Kuang, Y. C., Ooi, M. P., & Khoo, J. J. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2017.12.004 · Cited by 27
Evaluation of lung and chest wall mechanics during anaesthesia using the PEEP-step method
Persson, P., Stenqvist, O., & Lundin, S. (2017) · British Journal of Anaesthesia · doi:10.1016/j.bja.2017.11.076 · Cited by 27
Czycholl, I., Büttner, K., Klingbeil, P., & Krieter, J. (2018) · Animals · doi:10.3390/ani8010007 · Cited by 27
Gillen, Z. M., Miramonti, A. A., McKay, B. D., Leutzinger, T. J., & Cramer, J. T. (2018) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000002498 · Cited by 28
Saggino, A., Balsamo, M., Carlucci, L., Cavalletti, V., Sergi, M. R., Fermo, G. D., Dèttore, D., Marsigli, N., Petruccelli, I., Pizzo, S., et al. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00312 · Cited by 26
Doll, H., Gentile, B., Bush, E. N., & Ballinger, R. (2018) · Value in Health · doi:10.1016/j.jval.2018.02.006 · Cited by 26
Čelar, A., Tafaj, E., Graf, A. C., et al. (2018) · Journal of Orofacial Orthopedics / Fortschritte der Kieferorthopädie · doi:10.1007/s00056-018-0139-z · Cited by 26
Prediction of Shunt Dependency After Intracerebral Hemorrhage and Intraventricular Hemorrhage
Kuo, L., Lu, H.-Y., Tsai, J., & Tu, Y. (2018) · Neurocritical Care · doi:10.1007/s12028-018-0532-x · Cited by 26
McLachlan, K., Gray, A. L., Roesch, R., Douglas, K., & Viljoen, J. L. (2018) · Psychological Assessment · doi:10.1037/pas0000612 · Cited by 24
Mindful infrastructure as antecedent of innovation resilience behaviour of project teams
Oeij, P. R. A., van Vuuren, T., Dhondt, S., Gaspersz, J. B. R., & de Vroome, E. M. M. (2018) · Team Performance Management · doi:10.1108/tpm-09-2017-0045 · Cited by 29
Schmitt, A., Gielnik, M. M., & Seibel, S. (2018) · Motivation and Emotion · doi:10.1007/s11031-018-9720-4 · Cited by 28
Weyns, T., Colpin, H., Engels, M., Doumen, S., & Verschueren, K. (2019) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2018.12.009 · Cited by 32
Fernández‐Lázaro, G., Latorre, R., García, E. A., & Barja, I. (2019) · Behavioural Processes · doi:10.1016/j.beproc.2019.103897 · Cited by 26
The Neural Dynamics of Individual Differences in Episodic Autobiographical Memory
Petrican, R., Palombo, D. J., Sheldon, S., & Levine, B. (2020) · eNeuro · doi:10.1523/eneuro.0531-19.2020 · Cited by 27
PET quantification of brain O-GlcNAcase with [18F]LSN3316612 in healthy human volunteers
Lee, J., Liow, J., Paul, S., Morse, C. L., Haskali, M. B., Manly, L. S., Shcherbinin, S., Ruble, J. C., Kant, N., Collins, E. C., et al. (2020) · EJNMMI Research · doi:10.1186/s13550-020-0616-4 · Cited by 26
Brain Morphometry Estimation: From Hours to Seconds Using Deep Learning
Rebsamen, M., Suter, Y., Wiest, R., Reyes, M., & Rummel, C. (2020) · Frontiers in Neurology · doi:10.3389/fneur.2020.00244 · Cited by 26
Yan, S., Li, R., Shi, B., et al. (2020) · Gait & Posture · doi:10.1016/j.gaitpost.2020.05.007 · Cited by 26
Tosic, L., Goldberger, E., Maldaner, N., Sosnova, M., Zeitlberger, A. M., Staartjes, V. E., Gadjradj, P. S., Eversdijk, H. A. J., Quddusi, A. I., Gandía-González, M. L., et al. (2020) · Journal of Neurosurgery Spine · doi:10.3171/2020.3.spine2084 · Cited by 26
Germonpré, P., Papadopoulou, V., Hemelryck, W., Obeid, G., Lafère, P., Eckersley, R. J., Tang, M., & Balestra, C. (2014) · PubMed · Cited by 24
Christidi, F., Zalonis, I., Kyriazi, S., Rentzos, M., Karavasilis, E., Wilde, E. A., & Evdokimidis, I. (2013) · Brain Imaging and Behavior · doi:10.1007/s11682-013-9271-y · Cited by 26
Investigating placebo effects in irritable bowel syndrome: A novel research design
Conboy, L. A., Wasserman, R. H., Jacobson, E., Davis, R. B., Legedza, A., Park, M., Rivers, A. L., Morey, E. B., Nam, B. H., Lasagna, L. C., et al. (2006) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.11.012 · Cited by 26
Campbell, B., & Mark, M. M. (2006) · Journal of Applied Social Psychology · doi:10.1111/j.0021-9029.2006.00131.x · Cited by 26
Quantitative Evaluation of Video Laryngostroboscopy: Reliability of the Basic Parameters
Uloza, V., Vegienė, A., Pribuišienė, R., & Šaferis, V. (2013) · Journal of Voice · doi:10.1016/j.jvoice.2012.12.007 · Cited by 26
Nagasawa, Y., Demura, S., & Kitabayashi, T. (2004) · Perceptual and Motor Skills · doi:10.2466/pms.98.2.551-560 · Cited by 26
Clinical assessment of external rotation for the diagnosis of anterior shoulder hyperlaxity
ROPARS, M., Fournier, A., Campillo‐Gimenez, B., Bonan, I., Delamarche, P., Crétual, A., & Thomazeau, H. (2010) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2010.09.002 · Cited by 26
Hubbard, M., MacDermid, J. C., Kramer, J. F., et al. (2004) · Journal of Hand Therapy · doi:10.1197/j.jht.2003.10.004 · Cited by 26
Klein, M. G., Whyte, J., Keenan, M. A., Esquenazi, A., & Polansky, M. (2000) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(00)90113-8 · Cited by 26
The Validity of a Simple Outcome Measure to Assess Stuttering Therapy
Huinck, W. J., & Rietveld, T. C. M. (2007) · Folia Phoniatrica et Logopaedica · doi:10.1159/000098342 · Cited by 26
A Comparison of Two Versions of the Cognitive Therapy Scale
Gordon, P. K. (2006) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465806003390 · Cited by 26
Narcissistic needs of the self and perceptions of interpersonal behavior.
Robbins, S. B., & Dupont, P. (1992) · Journal of Counseling Psychology · doi:10.1037/0022-0167.39.4.462 · Cited by 26
Psychopathology and Competence to Stand Trial
Rosenfeld, B. D., & Wall, A. D. (1998) · Criminal Justice and Behavior · doi:10.1177/0093854898025004003 · Cited by 26
Preattentive Orienting in Adolescents With Mental Retardation
Merrill, E. C. (2004) · American Journal on Mental Retardation · doi:10.1352/0895-8017(2005)110 · Cited by 7
Reliability analysis of time series force plate data of community dwelling older adults
Bauer, C. M., Gröger, I., Rupprecht, R., Meichtry, A., Tibesku, C. O., & Gaßmann, K. (2010) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2010.01.009 · Cited by 26
Connolly, M. B., Crits-Christoph, P. F., Shappell, S., Barber, J. P., & Luborsky, L. B. (1998) · PubMed Central · Cited by 26
Computerized Platform Posturography for Children
Gabriel, L. S., & Mu, K. (2002) · Physical & Occupational Therapy In Pediatrics · doi:10.1080/j006v22n03_07 · Cited by 26
Determining the Occupational Reinforcer Patterns for O*NET Occupational Units
Wall, J. E., Rivkin, D. Β., & Lewis, P. (1999) · Cited by 26
Interobserver relational agreement.
Stine, W. W. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.106.2.341 · Cited by 26
Gruet, M., Brisswalter, J., Mély, L., & Vallier, J. (2010) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2009.12.008 · Cited by 25
Eurelings-Bontekoe, E. H. M., Onnink, A., Williams, M. M., et al. (2007) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2007.03.002 · Cited by 26
Long‐term course and outcome in unipolar affective and schizoaffective psychoses
Opjordsmoen, S. (1989) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1989.tb10265.x · Cited by 26
Interobserver reliability of methods for paleopathological diagnosis of dental caries
Rudney, J. D., Katz, R. V., & Brand, J. W. (1983) · American Journal of Physical Anthropology · doi:10.1002/ajpa.1330620303 · Cited by 26
Reproducibility of posterior capsule opacification measurement using Scheimpflug videophotography
Hayashi, K., Hayashi, H., Nakao, F., et al. (1998) · Journal of Cataract & Refractive Surgery · doi:10.1016/s0886-3350(98)80355-6 · Cited by 26
School Climate Reports from Norweginan Teachers: A Methodological and Substantive Study
Kallestad, J. H., Olweus, D., & Alsaker, F. D. (1998) · School Effectiveness and School Improvement · doi:10.1080/0924345980090104 · Cited by 26
Oxenham, D., Sheard, C., & Rogeradams (1995) · Aphasiology · doi:10.1080/02687039508248710 · Cited by 26
Holmbäck, A. M. (2001) · Journal of Rehabilitation Medicine · doi:10.1080/165019701750098966 · Cited by 26
Interpersonal processes and psychopathology among expectant and nonexpectant adolescent couples.
Moore, D. R., & Florsheim, P. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.69.1.101 · Cited by 26
Dumas, J. E. (1996) · Journal of Clinical Child Psychology · doi:10.1207/s15374424jccp2501_12 · Cited by 26
Comparison of Four Wisconsin Card Sorting Test Scoring Guides with Novice Raters
Axelrod, B. N., Greve, K. W., & GOLDMAN, R. S. (1994) · Assessment · doi:10.1177/1073191194001002001 · Cited by 26
Gwet, K. L. (2014) · Wiley eBooks · doi:10.1002/9781118445112.stat06882 · Cited by 31
Croxford, E., Gao, Y., First, E., Pellegrino, N., Schnier, M., Caskey, J., Oguss, M., Wills, G., Chen, G., Dligach, D., et al. (2025) · doi:10.1101/2025.04.22.25326219 · Cited by 25
The Decision to Adopt Evidence-Based and Other Innovative Mental Health Practices: Risky Business?
Panzano, P. C., & Roth, D. (2006) · Psychiatric Services · doi:10.1176/appi.ps.57.8.1153 · Cited by 115
IDEST: International Database of Emotional Short Texts
Kaakinen, J. K., Werlen, E., Kammerer, Y., Acartürk, C., Aparicio, X., Thierry Baccino, B., Ballenghein, U., Bergamin, P. B., Castells, N., Costa, A., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274480 · Cited by 26
Onan, D., Ülger, Ö., & Martelletti, P. (2023) · Expert Review of Neurotherapeutics · doi:10.1080/14737175.2023.2192870 · Cited by 24
Metastability as a candidate neuromechanistic biomarker of schizophrenia pathology
Hancock, F., Rosas, F. E., McCutcheon, R. A., Cabral, J., Dipasquale, O., & Turkheimer, F. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282707 · Cited by 25
Sacramento, C. A., Lyubovnikova, J., Martinaitytė, I., Gomes, C., Curral, L. A., & Juhasz‐Wrench, A. (2023) · Journal of Product Innovation Management · doi:10.1111/jpim.12699 · Cited by 30
Arizmendi, C., Wang, S., Kaplan, S. J., & Weinfurt, K. P. (2024) · Value in Health · doi:10.1016/j.jval.2024.01.016 · Cited by 26
Earl, E. H., Goyal, M. K., Mishra, S., Kannan, B., Mishra, A., Chowdhury, N., & Mishra, P. (2024) · Clinical Neurophysiology · doi:10.1016/j.clinph.2024.05.017 · Cited by 25
Savardi, M., Signoroni, A., Benini, S., Vaccher, F., Alberti, M., Ciolli, P., Di Meo, N., Falcone, T., Ramanzin, M., Romano, B., et al. (2025) · Insights into Imaging · doi:10.1186/s13244-024-01893-4 · Cited by 30
A new method to assess the influence of textiles properties on human thermophysiology. Part I
Annaheim, S., Wang, L.-C., Psikuta, A., et al. (2015) · International Journal of Clothing Science and Technology · doi:10.1108/ijcst-02-2014-0020 · Cited by 25
Bickelhaupt, S., Cattin, R., Froehlich, J. M., Raible, S., Bouquet, H., Bill, U., & Patak, M. A. (2013) · Magnetic Resonance in Medicine · doi:10.1002/mrm.24708 · Cited by 24
Jugular Venous Flow Abnormalities in Multiple Sclerosis Patients Compared to Normal Controls
Sethi, S. K., Utriainen, D. T., Daugherty, A. M., et al. (2014) · Journal of Neuroimaging · doi:10.1111/jon.12183 · Cited by 25
Effect of inversion and ankle bracing on peroneus longus Hoffmann reflex
Sefton, J. M., Hicks-Little, C. A., Koceja, D. M., & Cordova, M. L. (2006) · Scandinavian Journal of Medicine and Science in Sports · doi:10.1111/j.1600-0838.2006.00593.x · Cited by 25
Development of a measure of drinking and driving expectancies for youth.
McCarthy, D. M., Pedersen, S. L., Thompsen, D. M., & Leuty, M. E. (2006) · Psychological Assessment · doi:10.1037/1040-3590.18.2.155 · Cited by 25
Granzyme B ELISPOT assay to measure influenza-specific cellular immunity
Salk, H. M., Haralambieva, I. H., Ovsyannikova, I. G., Goergen, K. M., & Poland, G. A. (2013) · Journal of Immunological Methods · doi:10.1016/j.jim.2013.09.007 · Cited by 25
Wang, Q., & Yuan, Y. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.09.032 · Cited by 25
Zanto, T. P., Pa, J., & Gazzaley, A. H. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.063 · Cited by 26
Mehta, S. P., MacDermid, J. C., Richardson, J. A., MacIntyre, N. J., & Grewal, R. S. (2014) · Journal of Hand Therapy · doi:10.1016/j.jht.2014.09.009 · Cited by 26
The Validity and Reliability of Preoperative Radiographic Canal Diameter Measurements of the Femur
Walsh, K. P., Fowler, J. R., Chen, O., et al. (2013) · HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery · doi:10.1007/s11420-013-9334-z · Cited by 25
Intensity Estimation of Spontaneous Facial Action Units Based on Their Sparsity Properties
Mohammadi, M. R., Fatemizadeh, E., & Mahoor, M. H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2416317 · Cited by 25
Ranking Hepatologists: Which Hirsch's h-Index to prevent the “e-crise de foi-e”?
Poynard, T., Thabut, D., Jabre, P., Munteanu, M., Ratziu, V., Benhamou, Y., & Deckmyn, O. (2011) · Clinics and Research in Hepatology and Gastroenterology · doi:10.1016/j.clinre.2011.02.014 · Cited by 25
Risser patient satisfaction scale: a validation study in Greek cancer patients
Charalambous, A., & Αδαμακίδου, Θ. (2012) · BMC Nursing · doi:10.1186/1472-6955-11-27 · Cited by 25
Occlusive Dressings and the Healing of Standardized Abrasions
Beam, J. W. (2008) · Journal of Athletic Training · doi:10.4085/1062-6050-43.6.600 · Cited by 25
Comparison of Lifts Versus Tape Measure in Determining Leg Length Discrepancy
Badii, M., Wade, A., Collins, D. R., Nicolaou, S., Kobza, B. J., & Kopec, J. A. (2014) · The Journal of Rheumatology · doi:10.3899/jrheum.131089 · Cited by 25
Comparing the reliability of voluntary and evoked muscle actions
Jenkins, N. D. M., Palmer, T. B., & Cramer, J. T. (2013) · Clinical Physiology and Functional Imaging · doi:10.1111/cpf.12113 · Cited by 25
Fitzpatrick, J. P., Latimer, J., Ferreira, M. L., et al. (2013) · BMC Pediatrics · doi:10.1186/1471-2431-13-33 · Cited by 25
Wada, K., Mikami, H., Hamada, D., et al. (2016) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-015-2402-8 · Cited by 25
The radiodensity of cerebrospinal fluid and vitreous humor as indicator of the time since death
Koopmanschap, D. H. J. L. M., Bayat, A. R., Kubat, B., et al. (2016) · Forensic Science Medicine and Pathology · doi:10.1007/s12024-016-9778-9 · Cited by 25
Waldon, J., Begum, E. A., Gendron, M. A., Rusak, B., Andreou, P., Rajda, M., & Corkum, P. V. (2016) · Journal of Sleep Research · doi:10.1111/jsr.12402 · Cited by 25
Group interaction sustains positive moods and diminishes negative moods.
Park, E. S., & Hinsz, V. B. (2015) · Group Dynamics Theory Research and Practice · doi:10.1037/gdn0000034 · Cited by 25
Length of stay, therapeutic change, and recidivism for incarcerated juvenile offenders
Walker, S. C., & Bishop, A. S. (2016) · Journal of Offender Rehabilitation · doi:10.1080/10509674.2016.1194946 · Cited by 25
Cheng, C., Martin‐Biggers, J. T., Quick, V., Spaccarotella, K., & Byrd‐Bredbenner, C. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0417-3 · Cited by 24
Development and Testing of the Toronto Nutrition Environment Measures Survey–Store (ToNEMS-S)
Lo, B. K., Minaker, L. M., Mah, C. L., et al. (2016) · Journal of Nutrition Education and Behavior · doi:10.1016/j.jneb.2016.07.020 · Cited by 25
Dynamic contrast-enhanced MRI measurement of renal function in healthy participants
Eikefjord, E., Andersen, E., Hodneland, E., et al. (2016) · Acta Radiologica · doi:10.1177/0284185116666417 · Cited by 25
Meldrum, S., Savarimuthu, B. T. R., Licorish, S. A., Tahir, A., Bosu, M. F., & Jayakaran, P. (2017) · Digital Health · doi:10.1177/2055207617698908 · Cited by 25
Bronchopulmonary dysplasia-associated pulmonary hypertension: clues from placental pathology
Kunjunju, A., Gopagondanahalli, K. R., Chan, Y. F., et al. (2017) · Journal of Perinatology · doi:10.1038/jp.2017.130 · Cited by 25
Grooten, W. J. A., Sandberg, L., Ressman, J., Diamantoglou, N., Johansson, E., & Rasmussen-Barr, E. (2018) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-017-1927-0 · Cited by 25
Perceptual Characterization of the Macronutrient Picture System (MaPS) for Food Image fMRI
King, J. L., Fearnbach, S. N., Ramakrishnapillai, S., Shankpal, P., Geiselman, P. J., Martin, C. K., Murray, K. B., Hicks, J. L., McClernon, F. J., Apolzan, J. W., et al. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00017 · Cited by 26
Wenzel, F., Meyer, C., Stehlé, T., Peters, J., Gellißen, S., Thaler, C. P., & Zagorchev, L. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.03.001 · Cited by 25
Binkley, J. M., Stratford, P. W., Kirkpatrick, S. H., Farley, C. R., Okoli, J., & Gabram, S. (2018) · Clinical Breast Cancer · doi:10.1016/j.clbc.2018.02.008 · Cited by 26
Schumacher, L. V., Reisert, M., Nitschke, K., Egger, K. E., Urbach, H., Hennig, J., Weiller, C., & Kaller, C. P. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.01.086 · Cited by 25
Johannsen, A. M., Cook, A., Gardner, M. J., et al. (2018) · Clinical Anatomy · doi:10.1002/ca.23196 · Cited by 25
A System for the Measurement of the Subjective Visual Vertical using a Virtual Reality Device
Negrillo-Cárdenas, J., Rueda, A. J., Ogáyar, C. J., Lomas‐Vega, R., & Segura, R. J. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-0981-y · Cited by 26
Maternal Responsiveness to Gestures in Children With Down Syndrome
Lorang, E. K., Sterling, A. M., & Schroeder, B. (2018) · American Journal of Speech-Language Pathology · doi:10.1044/2018_ajslp-17-0138 · Cited by 25
Locascio, M., Infurna, M. R., Guarnaccia, C., Mancuso, L., Bifulco, A. T., & Giannone, F. (2018) · Journal of Interpersonal Violence · doi:10.1177/0886260518794512 · Cited by 25
Influence of the Box Dimensions on the Reliability and Validity of the Sit and Reach in Preschoolers
Pérez, C. A., Pérez, S. Á., Baamonde, S. G., et al. (2018) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000002886 · Cited by 25
Reynolds, P. S., Fisher, B. J., McCarter, J., Sweeney, C., Martin, E. J., Middleton, P., Ellenberg, M., Fowler, E., Brophy, D. F., Fowler, A. A., et al. (2018) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000001844 · Cited by 25
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
