The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 6,701–6,800 of 23,882
Initial reliability and validity studies of the revised Treatment Services Review (TSR-6).
Cacciola, J., Alterman, A., Lynch, K., Martin, J. M., Beauchamp, M., & Mclellan, A. (2008) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2007.06.004 · Cited by 42
Avi Besser, A., & Priel, B. (2011) · Psychiatry · doi:10.1521/psyc.2011.74.1.31 · Cited by 41
Test-Retest Reliability of the Progressive Isoinertial Lifting Evaluation (PILE)
Lygren, H., Dragesund, T., Joensen, J., et al. (2005) · Spine · doi:10.1097/01.brs.0000160850.51550.55 · Cited by 41
Test–retest reliability of the Tower of London Planning Task (TOL-F).
Schumacher, L. V., Nitschke, K., Schumacher, F. K., Weiller, C., & Kaller, C. P. (2015) · Psychological Assessment · doi:10.1037/pas0000097 · Cited by 41
Cognitive correlates of schizophrenia signs and symptoms: I. verbal communication disturbances
Berenbaum, H., Kerns, J. G., Vernon, L. L., & Gómez, J. J. B. (2008) · Psychiatry Research · doi:10.1016/j.psychres.2007.08.016 · Cited by 41
A Rorschach Stability Study in a Nonpatient Adult Sample
Sultan, S., Andronikof, A., Réveillère, C., & Lemmel, G. (2006) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8703_13 · Cited by 41
Susser, S. R., McCusker, J., & Belzile, É. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.07.009 · Cited by 41
FACTORS AFFECTING RELIABILITY OF INTERPRETATIONS OF SCREE PLOTS
Streiner, D. L. (1998) · Psychological Reports · doi:10.2466/pr0.1998.83.2.687 · Cited by 41
Schuringa, E., Spreen, M., & Bogaerts, S. (2014) · Journal of Forensic Psychology Practice · doi:10.1080/15228932.2014.897536 · Cited by 41
Liu, M., Toikawa, H., Seki, M., et al. (1998) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/00002060-199801000-00006 · Cited by 41
The reliability of production strategy scores for the Ruff Figural Fluency Test
Ross, T. P., Foard, E., Hiott, F. B., et al. (2003) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/18.8.879 · Cited by 41
Cicchetti, D. V., Showalter, D. R., & Rosenheck, R. A. (1997) · Psychiatry Research · doi:10.1016/s0165-1781(97)00095-4 · Cited by 41
Zammit, G. V., & Payne, C. (2007) · Journal of the American Podiatric Medical Association · doi:10.7547/0970207 · Cited by 41
von Consbruch, K., Clark, D. M., & Stangier, U. (2011) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465811000622 · Cited by 43
Reduction of the Internal Capsule in Families Affected with Schizophrenia
Wobrock, T., Kamer, T., Roy, A., Vogeley, K. T., Schneider-Axmann, T., Wagner, M., Maier, W., Rietschel, M., Schulze, T. G., Scherk, H., et al. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2007.02.026 · Cited by 42
van Meeteren, J., van Rijn, R. M., Selles, R. W., et al. (2007) · Acta Dermato Venereologica · doi:10.2340/16501977-0095 · Cited by 41
Zima, B. T., Bussing, R., Crecelius, G. M., Kaufman, A., & Belin, T. R. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.11.1732 · Cited by 41
Kaminski, T. W., Groff, R. M., & Glutting, J. J. (2009) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803390802484771 · Cited by 41
Gajewski, B. J., Hart, S., Bergquist‐Beringer, S., & Dunton, N. E. (2007) · Statistics in Medicine · doi:10.1002/sim.2877 · Cited by 41
A New Tool for Automatic Assessment of Segmental Wall Motion Based on Longitudinal 2D Strain
Liel‐Cohen, N., Tsadok, Y., Beeri, R., Lysyansky, P., Agmon, Y., Feinberg, M. S., Fehske, W., Gilon, D., Hay, I., Kuperstein, R., et al. (2009) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.108.841874 · Cited by 42
Verhaegen, P. D. H. M., van Marle, J., Kuehne, A., Schouten, H. J., Gaffney, E. A., Maini, P. K., Middelkoop, E., & van Zuijlen, P. P. M. (2011) · Journal of Microscopy · doi:10.1111/j.1365-2818.2011.03547.x · Cited by 41
Bias in White: A Longitudinal Natural Experiment Measuring Changes in Discrimination
Rubineau, B., & Kang, Y. (2011) · Management Science · doi:10.1287/mnsc.1110.1439 · Cited by 42
Parkinson disease, brain volumes, and subthalamic nucleus stimulation
Bonneville, F., Welter, M., Elie, C., du Montcel, S. T., Hasboun, D., Guillevin, C., Houéto, J., Bonnet, A. M., Mesnage, V., Pidoux, B., et al. (2005) · Neurology · doi:10.1212/01.wnl.0000160401.24880.83 · Cited by 41
Rater Agreement on Gait Assessment during Neurologic Examination of Horses
Olsen, E., Dunkel, B., Barker, W. H. J., Finding, E., PERKINS, J. D., Witte, T. H., Yates, L. J., Andersen, P. H., Baiker, K., & Piercy, R. J. (2014) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.12320 · Cited by 44
Byl, T., Cole, J. A., & Livingston, L. A. (2000) · Journal of Sport Rehabilitation · doi:10.1123/jsr.9.1.26 · Cited by 41
OPTOTRAK Measurement of the Quadriceps Angle Using Standardized Foot Positions.
Livingston, L. A., & Spaulding, S. J. (2002) · PubMed · Cited by 41
Validity and reliability of grade scoring in the diagnosis of exercise-induced laryngeal obstruction
Walsted, E. S., Hull, J. H., Hvedstrup, J., Maat, R. C., & Backer, V. (2017) · ERJ Open Research · doi:10.1183/23120541.00070-2017 · Cited by 42
Nunes, E., Goehl, L., Seracini, A. M., Deliyannides, D. A., Donovan, S., Koenig, T., Quitkin, F., & Williams, J. B. W. (1996) · American Journal on Addictions · doi:10.1111/j.1521-0391.1996.tb00307.x · Cited by 42
Differential strength decline in patients with osteoarthritis of the knee: revision of a hypothesis
Hall, K. D., Hayes, K. W., & Falconer, J. (1993) · Arthritis & Rheumatism · doi:10.1002/art.1790060208 · Cited by 41
McMahon, L. M., Burdett, R. G., & Whitney, S. L. (1992) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.1992.15.5.236 · Cited by 41
O’Brian, N. R., O’Brian, S., Packman, A., et al. (2003) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2003/056 · Cited by 41
Preferences for Female Body Weight and Shape in Three European Countries
Swami, V., Neto, F. F. M., Tovée, M. J., & Furnham, A. F. (2007) · European Psychologist · doi:10.1027/1016-9040.12.3.220 · Cited by 40
Changes in strength over time among polio survivors
Klein, M. G., Whyte, J., Keenan, M. A., et al. (2000) · Archives of Physical Medicine and Rehabilitation · doi:10.1053/apmr.2000.3890 · Cited by 41
Suitability for brief dynamic psychotherapy: psychodynamic variables as predictors of outcome
Høglend, P. A. (1993) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1993.tb03422.x · Cited by 41
Selective attention in schizophrenia: relationship to verbal working memory
Barch, D. M., & Carter, C. S. (1998) · Schizophrenia Research · doi:10.1016/s0920-9964(98)00064-4 · Cited by 41
Barber, J. J., McDermott, A. Y., McGaughey, K. J., Olmstead, J. D., & Hagobian, T. A. (2013) · Journal of Strength and Conditioning Research · doi:10.1519/jsc.0b013e318252f6b7 · Cited by 40
Sex differences in amygdala subregions: evidence from subregional shape analysis
Kim, H. J., Kim, N., Kim, S., Hong, S., Park, K., Lim, S., Park, J.-M., Na, B., Chae, Y., Lee, J., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.025 · Cited by 40
Donohue, C., Khalifa, Y., Perera, S., Sejdić, E., & Coyle, J. L. (2021) · Dysphagia · doi:10.1007/s00455-020-10177-0 · Cited by 41
Simpson, P. A., Mars, T., & Esteves, J. E. (2017) · International Journal of Osteopathic Medicine · doi:10.1016/j.ijosm.2017.03.001 · Cited by 38
Looney, P., Yin, Y., Collins, S. L., Nicolaides, K. H., Plasencia, W., Molloholli, M., Natsis, S., & Stevenson, G. N. (2021) · IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control · doi:10.1109/tuffc.2021.3052143 · Cited by 40
Neck sensor-supported hyoid bone movement tracking during swallowing
Mao, S., Zhang, Z., Khalifa, Y., Donohue, C., Coyle, J. L., & Sejdic, E. (2019) · Royal Society Open Science · doi:10.1098/rsos.181982 · Cited by 47
What gets in the way of social engagement in schizophrenia?
Weittenhiller, L. P., Mikhail, M. E., Mote, J., Campellone, T. R., & Kring, A. M. (2021) · World Journal of Psychiatry · doi:10.5498/wjp.v11.i1.13 · Cited by 45
A comprehensive empirical study on bug characteristics of deep learning frameworks
Yang, Y., He, T., Xia, Z., & Feng, Y. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107004 · Cited by 41
More Voices Persuade: The Attentional Benefits of Voice Numerosity
Chang, H. H., Mukherjee, A., & Chattopadhyay, A. (2022) · Journal of Marketing Research · doi:10.1177/00222437221134115 · Cited by 40
Plantar foot pressures after the augmented low dye taping technique.
Vicenzino, B., McPoil, T. G., & Buckland, S. (2008) · PubMed · Cited by 40
Mäntylä, T. J. S., Mantere, O., Raij, T. T., Kieseppä, T., Laitinen, H. K., Leiviskä, J., Torniainen, M., Tuominen, L. J., Vaarala, O., & Suvisaari, J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0125112 · Cited by 42
Reeves, M. M., WINKLER, E. A. H., & Eakin, E. (2014) · Nutrition & Dietetics · doi:10.1111/1747-0080.12160 · Cited by 42
Fawzi, M. H., Fawzi, M. M., & Abu-Hindi, W. (2012) · Eastern Mediterranean Health Journal · doi:10.26719/2012.18.4.304 · Cited by 40
The effect of intense physical therapy for children with cerebral palsy
Christy, J. B., Chapman, C., & Murphy, P. (2012) · Journal of Pediatric Rehabilitation Medicine · doi:10.3233/prm-2012-0208 · Cited by 40
Aytar, A., Yürük, Z. Ö., Tüzün, E. H., et al. (2015) · Journal of Back and Musculoskeletal Rehabilitation · doi:10.3233/bmr-140545 · Cited by 40
Temporal reliability of cytokines and growth factors in EDTA plasma
Clendenen, T. V., Arslan, A. A., Lokshin, A. E., Idahl, A., Hallmans, G., Koenig, K. L., Marrangoni, A., Nolen, B. M., Ohlson, N., Zeleniuch‐Jacquotte, A., et al. (2010) · BMC Research Notes · doi:10.1186/1756-0500-3-302 · Cited by 42
Baets, L. D., Jaspers, E., Desloovere, K., & Deun, S. V. (2012) · Journal of Electromyography and Kinesiology · doi:10.1016/j.jelekin.2012.06.007 · Cited by 40
Dhiman, N., Ovsyannikova, I. G., Ryan, J. E., et al. (2005) · Clinical & Experimental Immunology · doi:10.1111/j.1365-2249.2005.02931.x · Cited by 40
Epigenetic Changes in Basal Cell Carcinoma Affect SHH and WNT Signaling Components
Brinkhuizen, T., van den Hurk, K., Winnepenninckx, V. J. L., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0051710 · Cited by 40
Nardo, L., Parimi, N., Liu, F., et al. (2015) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-015-4222-0 · Cited by 40
Noninvasive measurement of systemic vascular resistance using Doppler echocardiography
Abbas, A. E., Fortuin, F. D., Patel, B. M., et al. (2004) · Journal of the American Society of Echocardiography · doi:10.1016/j.echo.2004.04.008 · Cited by 40
Psychometric Evaluation of the Life Orientation Test—Revised in Treated Opiate Dependent Individuals
Hirsch, J. K., Britton, P. C., & Conner, K. R. (2009) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-009-9224-2 · Cited by 40
Nicastro, R., Jermann, F., Bondolfi, G., & McQuillan, A. (2010) · Assessment · doi:10.1177/1073191110363551 · Cited by 46
An anatomical study of the entry point in the greater trochanter for intramedullary nailing
Farhang, K., Desai, R. G., Wilber, J. H., et al. (2014) · The Bone & Joint Journal · doi:10.1302/0301-620x.96b9.34314 · Cited by 40
A case-control study of determinants for high and low dental caries prevalence in Nevada youth
Ditmyer, M. M., Dounis, G., Mobley, C. C., et al. (2010) · BMC Oral Health · doi:10.1186/1472-6831-10-24 · Cited by 40
Albrecht, U., von Jan, U., Sedlacek, L., Groos, S., Suerbaum, S., & Vonberg, R. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2643 · Cited by 40
Kouvelioti, V., Kellis, E., Kofotolis, N. D., et al. (2015) · Research in Sports Medicine · doi:10.1080/15438627.2015.1005292 · Cited by 40
Malone, D. C., Abarca, J., Hansten, P. D., Grizzle, A. J., Armstrong, E. P., Bergen, R. C. V., Duncan, B. S., Solomon, S. L., & Lipton, R. B. (2005) · The American Journal of Geriatric Pharmacotherapy · doi:10.1016/j.amjopharm.2005.05.001 · Cited by 40
Vertriest, S., Coorevits, P. L., Hagberg, K., Brånemark, R., Häggström, E., Vanderstraeten, G., & Frossard, L. A. (2014) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2014.2337956 · Cited by 40
Using ultrasound imaging to identify landmarks in vertebra models to assess spinal deformity
Chen, W., Lou, E. H. M., & Le, L. H. (2011) · doi:10.1109/iembs.2011.6092096 · Cited by 40
Givens, D. L., Fitch, D. S., Fritz, J. M., et al. (2004) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2004.34.8.430 · Cited by 40
Psychometric evaluation of the Psychotic Symptom Rating Scales
Kronmüller, K., von Bock, A., Grupe, S., Büche, L., Gentner, N. C., Rückl, S., Marx, J., Joest, K., Kaiser, S., Vedder, H., et al. (2010) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2010.04.014 · Cited by 43
Polycyclic Aromatic Hydrocarbon-DNA Adducts in Prostate Cancer
Rybicki, B. A., Rundle, A., Savera, A. T., et al. (2004) · Cancer Research · doi:10.1158/0008-5472.can-04-2323 · Cited by 40
Golriz, S., Hébert, J. J., Foreman, K. B., & Walker, B. F. (2012) · Chiropractic & Manual Therapies · doi:10.1186/2045-709x-20-14 · Cited by 41
Sun Protection at Elementary Schools: A Cluster Randomized Trial
Hunter, S., Love-Jackson, K. M., Abdulla, R. A., Zhu, W., Lee, J., Wells, K. J., & Roetzheim, R. G. (2010) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djq010 · Cited by 40
Collectivistic leadership and George C. Marshall: A historiometric analysis of career events
Friedrich, T. L., Vessey, W. B., Schuelke, M. J., Mumford, M. D., Yammarino, F. J., & Ruark, G. A. (2013) · The Leadership Quarterly · doi:10.1016/j.leaqua.2013.10.012 · Cited by 42
Mascitelli, A. N., Rojahn, J., Nicolaides, V., Moore, L., Hastings, R. P., & Christian‐Jones, C. (2015) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.12152 · Cited by 40
Occupational Asthma guidelines: a systematic quality appraisal using the AGREE II instrument
Lytras, T., Bonovas, S., Chronis, C., Konstantinidis, A. K., Kopsachilis, F., Papamichail, D. P., & Dounias, G. (2013) · Occupational and Environmental Medicine · doi:10.1136/oemed-2013-101656 · Cited by 40
Psychometric evaluation of ADAS-Cog and NTB for measuring drug response
Karin, A. A., Hannesdóttir, K., Jaeger, J., Annas, P., Segerdahl, M., Karlsson, P., Sjögren, N., von Rosen, T., & Miller, F. (2013) · Acta Neurologica Scandinavica · doi:10.1111/ane.12153 · Cited by 41
Shea, M. T., Reddy, M. K., Tyrka, A. R., & Sevin, E. (2013) · Psychiatry Research · doi:10.1016/j.psychres.2013.08.039 · Cited by 39
Togao, O., Hiwatashi, A., Keupp, J., Yamashita, K., Kikuchi, K., Yoshiura, T., Suzuki, Y., Kruiskamp, M. J., Sagiyama, K., Takahashi, M., et al. (2015) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.24895 · Cited by 39
The international Hip Outcome Tool-33 (iHOT-33): multicenter validation and translation to Spanish
Ibán, M. Á. R., Seijas, R., Sallent, A., et al. (2015) · Health and Quality of Life Outcomes · doi:10.1186/s12955-015-0255-z · Cited by 40
Affective Instability and Reactivity in Depressed Patients With and Without Borderline Pathology
Köhling, J., Moessner, M., Ehrenthal, J. C., Bauer, S., Cierpka, M., Kämmerer, A., Schauenburg, H., & Dinger, U. (2015) · Journal of Personality Disorders · doi:10.1521/pedi_2015_29_230 · Cited by 39
Belavý, D. L., Armbrecht, G., & Felsenberg, D. (2015) · Physiological Measurement · doi:10.1088/0967-3334/36/11/2285 · Cited by 40
Registering methodology for imaging and analysis of residual-limb shape after transtibial amputation
Dickinson, A., Steer, J. W., Woods, C. J., & Worsley, P. R. (2016) · The Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2014.10.0272 · Cited by 40
Premorbid retroversion is significantly greater in type B2 glenoids
Knowles, N. K., Ferreira, L. M., & Athwal, G. S. (2016) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2015.11.002 · Cited by 40
The role of valence focus and appraisal overlap in emotion differentiation.
Erbaş, Y., Ceulemans, E., Koval, P., & Kuppens, P. (2015) · Emotion · doi:10.1037/emo0000039 · Cited by 51
Mayorga-Vega, D., Escaño, J. M., Merino-Marbán, R., & Viciana, J. (2015) · European Physical Education Review · doi:10.1177/1356336x15599010 · Cited by 40
Lü, Y., Boudiffa, M., Dall’Ara, E., et al. (2016) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2016.05.022 · Cited by 40
The Quality and Accuracy of Mobile Apps to Prevent Driving After Drinking Alcohol
Wilson, H., Stoyanov, S. R., Gandabhai, S., & Baldwin, A. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5961 · Cited by 40
Prevost, L. B., Smith, M. K., & Knight, J. (2016) · CBE—Life Sciences Education · doi:10.1187/cbe.15-12-0267 · Cited by 40
Wang, T., Liang, K., Liu, Y.-C., Shieh, J., & Chen, H. (2017) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2017.01.024 · Cited by 41
Evidence for neuroplastic compensation in the cerebral cortex of persons with depressive illness
Bansal, R., Hellerstein, D. J., & Peterson, B. S. (2018) · Molecular Psychiatry · doi:10.1038/mp.2017.34 · Cited by 41
Recasens, M., & Uhlhaas, P. J. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.05.064 · Cited by 42
Decision qualities of Bayes factor and p value-based hypothesis testing.
Jeon, M., & de Boeck, P. A. L. (2017) · Psychological Methods · doi:10.1037/met0000140 · Cited by 44
Wang, Q., Tan, Q., Xu, W., et al. (2017) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2017.07.010 · Cited by 40
Chuang, L.-L., Chuang, Y.-F., Hsu, M., Huang, Y., Wong, M., & Chang, Y. (2018) · PLoS ONE · doi:10.1371/journal.pone.0189850 · Cited by 39
Kang, Y., Mozley, P. D., Verma, A. K., Schlyer, D. J., Henchcliffe, C., Gauthier, S. A., Chiao, P., He, B., Nikolopoulou, A., Logan, J., et al. (2018) · Journal of Neuroimaging · doi:10.1111/jon.12519 · Cited by 40
Factors Influencing Central Lamina Cribrosa Depth: A Multicenter Study
Luo, H., Yang, H., Gardiner, S. K., et al. (2018) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.17-23456 · Cited by 40
Hardy, J. H., Day, E. A., & Steele, L. M. (2018) · Journal of Management · doi:10.1177/0149206318780440 · Cited by 44
Faure, R., Righetti, F., Seibel, M., & Hofmann, W. (2018) · Psychological Science · doi:10.1177/0956797618785899 · Cited by 46
Mousavi, S. J., Tromp, R., Swann, M. C., White, A. P., & Anderson, D. (2018) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2018.08.033 · Cited by 40
Introducing a scalable peer feedback system for learning teams
O’Neill, T., Larson, N., Smith, J. L., Donia, M. B. L., Deng, C., Rosehart, W., & Brennan, R. W. (2018) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1526256 · Cited by 41
Valence, arousal, and dominance ratings for facial stimuli
Sutton, T. M., Herbert, A. M., & Clark, D. Q. (2019) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021819829012 · Cited by 40
Pieles, G. E., Grosse‐Wortmann, L., Hader, M., Fatah, M., Chungsomprasong, P., Slorach, C., Hui, W., Fan, C. S., Manlhiot, C., Mertens, L., et al. (2019) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.118.007693 · Cited by 40
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
