The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 13,901–14,000 of 23,881
Basma, H. S., Saleh, M. H. A., Geurs, N. C., et al. (2022) · Journal of Periodontology · doi:10.1002/jper.22-0212 · Cited by 11
Interrater Reliability of the Peer Review Process in Management Journals
Nicolai, A. T., Schmal, S., & Schuster, C. L. (2014) · Springer eBooks · doi:10.1007/978-3-319-09785-5_7 · Cited by 11
A Comparison of Incumbent and Analyst Ratings of O*NET Skills
Tsacoumis, S., & Iddekinge, C. H. V. (2006) · Cited by 11
Reliability and practicability of the straight leg raise test in children with cerebral palsy
Marsico, P., Tal‐Akabi, A., & van Hedel, H. J. A. (2015) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.12797 · Cited by 11
Byl, N. N., & Sadowsky, H. S. (1993) · Isokinetics and Exercise Science · doi:10.3233/ies-1993-3303 · Cited by 11
Assessing comorbidity in older adults using prescription claims data
Dubois, M., Dubuc, N., Kröger, E., Girard, R., & Hébert, R. (2010) · Journal of Pharmaceutical Health Services Research · doi:10.1111/j.1759-8893.2010.00030.x · Cited by 11
Fabbra, M., & Demarchi, D. A. (2013) · Cuadernos del Instituto Nacional de Antropología y Pensamiento Latinoamericano – Series Especiales · Cited by 11
Baer, G. D., Smith, M., Rowe, P. J., et al. (2003) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(03)00050-9 · Cited by 11
Assessing Stability between Treatment Planning Documents in a System of Care
肥後本, 芳., 上杉, 忍., Chorpita, B. F., et al. (2007) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0137-8 · Cited by 11
Facilitating information system development with panoramic view on data
Lavbič, D., Lajovic, I., & Krisper, M. (2010) · Computer Science and Information Systems · doi:10.2298/csis091122031l · Cited by 11
Shaw, S., & Coggins, T. E. (1991) · Journal of Speech Language and Hearing Research · doi:10.1044/jshr.3405.989 · Cited by 11
Dilatation of the Bile Duct in Patients after Cholecystectomy: A Retrospective Study
Landry, D., Tang, A., Murphy-Lavallée, J., Lepanto, L., Billiard, J.-S., Olivié, D., & Sylvestre, M. (2013) · Canadian Association of Radiologists Journal · doi:10.1016/j.carj.2012.09.004 · Cited by 12
Vandervelde, L., Dispa, D., Bergh, P. Y. V. D., et al. (2008) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2008.01.024 · Cited by 11
Measurement of Active Rotation in Standing: Reliability of a Simple Test Protocol
Evans, K. A., Refshauge, K. M., & Adams, R. D. (2006) · Perceptual and Motor Skills · doi:10.2466/pms.103.2.619-628 · Cited by 11
Predictors of change after long‐term analytic group psychotherapy
Lorentzen, S., & Høglend, P. A. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20207 · Cited by 11
Reproducibility and Repeatability of Volumetric Measurements for Olfactory Bulb Volumetry:
Burmeister, H. P., Baltzer, P., Möslein, C., Bitter, T., Gudziol, H., Dietzel, M., Guntinas‐Lichius, O., & Kaiser, W. A. (2011) · Academic Radiology · doi:10.1016/j.acra.2011.02.018 · Cited by 11
Interrater Reliability of the Record of Driving Errors (RODE)
Barco, P. P., Carr, D. B., Rutkoski, K., et al. (2015) · American Journal of Occupational Therapy · doi:10.5014/ajot.2015.013128 · Cited by 11
MEASURING LUMBAR SAGITTAL POSTURE IN SEWAGE WORKERS USING AN ULTRASONIC DEVICE
Friedrich, M. (2002) · Journal of Musculoskeletal Research · doi:10.1142/s0218957702000873 · Cited by 11
Matsuda, S., Demura, S., Kasuga, K., et al. (2012) · Advances in Physical Education · doi:10.4236/ape.2012.22008 · Cited by 11
Burkhart, T. A., Clarke, D., & Andrews, D. M. (2012) · Journal of Biomechanical Engineering · doi:10.1115/1.4005543 · Cited by 11
Caty, G. D., Theunissen, E., & Lejeune, T. M. (2009) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2008.12.008 · Cited by 11
Cohen, B. L., Tang, R. S. Y., Groessl, E. J., et al. (2012) · Journal of Interventional Gastroenterology · doi:10.4161/jig.23748 · Cited by 11
Boyer, F. C., Novella, J., Bertaud, S., Delmer, F., Vesselle, B., & Étienne, J. (2005) · Clinical Rehabilitation · doi:10.1191/0269215505cr858oa · Cited by 11
Hillege, S. L., de Ruiter, C., Smits, N., et al. (2011) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2011.631207 · Cited by 11
Key Dimensions and Validity of the Chinese Version of the Individualism-Collectivism Scale
Renzhi, H., Shuqiao, Y., Abela, J. R. Z., et al. (2013) · Chinese Studies · doi:10.4236/chnstd.2013.21001 · Cited by 11
Fristedt, S., Andersson, E. E., & Unsworth, C. A. (2013) · Scandinavian Journal of Occupational Therapy · doi:10.3109/11038128.2013.777940 · Cited by 11
Quantitative Evaluation of Facial Movement and Morphology
Vimercati, S. L., Rigoldi, C., Albertini, G., et al. (2012) · Annals of Otology Rhinology & Laryngology · doi:10.1177/000348941212100410 · Cited by 11
An Ordinal Coefficient of Relational Agreement for Multiple Judges
Fagot, R. F. (1994) · Psychometrika · doi:10.1007/bf02295186 · Cited by 11
Mathias, S. D., Crosby, R. D., Nazir, J., et al. (2014) · Value in Health · doi:10.1016/j.jval.2014.09.002 · Cited by 11
Pilot Instructor Rater Training: The Utility of the Multifacet Item Response Theory Model
Mulqueen, C., Baker, D. P., & Dismukes, R. K. (2002) · International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1203_6 · Cited by 11
Prenatal Cocaine Exposure and Motor Performance at 4 Months
Bigsby, R., LaGasse, L. L., Lester, B. M., et al. (2011) · American Journal of Occupational Therapy · doi:10.5014/ajot.2011.001263 · Cited by 11
Ye, R., & Wang, S.-G. (2008) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2008.05.004 · Cited by 11
Estimating Individual Rater Reliabilities
Overall, J. E., & Magee, K. N. (1992) · Applied Psychological Measurement · doi:10.1177/014662169201600109 · Cited by 11
Radia-George, C., Imms, C., & Taylor, N. F. (2014) · American Journal of Occupational Therapy · doi:10.5014/ajot.2014.009878 · Cited by 11
Obuchowski, N. A. (2001) · Statistics in Medicine · doi:10.1002/sim.929 · Cited by 11
Unsworth, C. A., Pallant, J., Russell, K. J., et al. (2011) · British Journal of Occupational Therapy · doi:10.4276/030802211x13125646370960 · Cited by 11
Taylor, N. F., Evans, O. M., & Goldie, P. A. (2001) · Physiotherapy Research International · doi:10.1002/pri.229 · Cited by 11
Visual judgements of steadiness in one‐legged stance: reliability and validity
Haupstein, T., & Goldie, P. A. (2000) · Physiotherapy Research International · doi:10.1002/pri.195 · Cited by 11
Observing physical activity in suburbs
Suminski, R. R., Jr., Fritzsinger, J., Leck, T. M., & Hyder, M. (2008) · Health & Place · doi:10.1016/j.healthplace.2007.12.004 · Cited by 11
Shin, H. Y., Park, H.-Y. L., Jung, K. I., et al. (2012) · Japanese Journal of Ophthalmology · doi:10.1007/s10384-012-0205-9 · Cited by 11
Perceived Social Support, Malevolent Maternal Representations, and Older Adults' Depressed Mood
Avi Besser, A., & Priel, B. (2007) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2007.26.6.728 · Cited by 11
Correlation analysis for longitudinal data: applications to HIV and psychosocial research
Tu, X. M., Feng, C., Kowalski, J., Tang, W., Wang, H., Wan, C., & Ma, Y. (2007) · Statistics in Medicine · doi:10.1002/sim.2857 · Cited by 11
Iezzi, R., Cotroneo, A. R., Filippone, A., et al. (2007) · American Journal of Roentgenology · doi:10.2214/ajr.06.1116 · Cited by 11
Allen, M. K., Wellen, M. A., Hart, D. P., et al. (2007) · Physiotherapy Canada · doi:10.3138/ptc.59.4.286 · Cited by 11
Cape, J. D., Morris, E., Burd, M., & Buszewicz, M. (2008) · British Journal of General Practice · doi:10.3399/bjgp08x299281 · Cited by 11
Self-assessment of Foot Health
Trevethan, R. (2009) · Journal of the American Podiatric Medical Association · doi:10.7547/0990460 · Cited by 11
Metsna, V., Sarap, P., Vorobjov, S., et al. (2013) · Acta Orthopaedica et Traumatologica Turcica · doi:10.3944/aott.2013.3079 · Cited by 11
Recovery from Depression Through Buddhist Wisdom: An Idiographic Case Study
Cheng, F. K. (2014) · Social Work in Mental Health · doi:10.1080/15332985.2014.891554 · Cited by 11
A four‐disc version of the Tower of London for clinical use
Tunstall, J. R., O’Gorman, J. G., & Shum, D. (2014) · Journal of Neuropsychology · doi:10.1111/jnp.12060 · Cited by 11
Lindgren, L., Gouveia-Figueira, S. C., Nording, M. L., & Fowler, C. J. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1450-z · Cited by 11
Sublingual tissue perfusion improves during emergency treatment of acute decompensated heart failure
Hogan, C., Ward, K. R., Franzen, D., et al. (2011) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2011.06.005 · Cited by 11
Borkenau, P., Leising, D., & Fritz, U. (2013) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000188 · Cited by 11
Lee, Y. S., Ahn, J. H., Lim, H. C., et al. (2008) · The American Journal of Sports Medicine · doi:10.1177/0363546508324691 · Cited by 11
Magyar, M. S., Carr, W. A., Rosenfeld, B. D., et al. (2009) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x09344105 · Cited by 11
Wojciszke, B., Pienkowski, R., Maroszek, A., et al. (1993) · European Journal of Social Psychology · doi:10.1002/ejsp.2420230304 · Cited by 11
Eubanks, D. L., Palanski, M. E., Swart, J., Hammond, M. M., & Olabisi, J. O. (2014) · The Journal of Creative Behavior · doi:10.1002/jocb.70 · Cited by 12
Ababneh, B. A., Rejjal, L., Pokharel, Y., et al. (2014) · Vascular Medicine · doi:10.1177/1358863x14549270 · Cited by 11
Ramchandren, S., Shy, M. E., Feldman, E. L., et al. (2014) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp-2013-307390 · Cited by 11
Moghadam, A. N., Mohammadi, R., Arab, A. M., et al. (2011) · PubMed · Cited by 11
Accuracy of haptic assessment of patellar symmetry in total knee arthroplasty
DeOrio, J. K., & Peden, J. P. (2004) · The Journal of Arthroplasty · doi:10.1016/j.arth.2003.12.066 · Cited by 11
Measuring Absorptive Capacity in High-Tech Companies
Sedoglavich, V., Akoorie, M. E. M., & Pavlovich, K. (2014) · Journal of Mixed Methods Research · doi:10.1177/1558689814523677 · Cited by 11
Sheppard, M. (2009) · The British Journal of Social Work · doi:10.1093/bjsw/bcn163 · Cited by 11
Frequency Content of Cartilage Impact Force Signal Reflects Acute Histologic Structural Damage
Heiner, A. D., Martin, J. A., McKinley, T. O., et al. (2012) · Cartilage · doi:10.1177/1947603511430706 · Cited by 11
López, F., & Thompson, S. S. (2010) · Reading & Writing Quarterly · doi:10.1080/10573561003769640 · Cited by 11
Development and Validation of the Employer Openness Survey
Gilbride, D. D., Vandergoot, D., Golden, K., & Stensrud, R. (2006) · Rehabilitation Counseling Bulletin · doi:10.1177/00343552060490020901 · Cited by 11
Cunningham, L. J., O’Rourke, K., Finlay, C., & Gallagher, M. (2015) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2015.1084334 · Cited by 13
Triple Jump Examinations for Dental Student Assessment
Navazesh, M., Rich, S. K., Chopiuk, N. B., & Keim, R. G. (2013) · Journal of Dental Education · doi:10.1002/j.0022-0337.2013.77.10.tb05605.x · Cited by 11
Bulstra, L. F., Hundepool, C. A., Friedrich, P. F., et al. (2015) · The Journal Of Hand Surgery · doi:10.1016/j.jhsa.2015.11.002 · Cited by 11
Grading Written Essays: A Reliability Study
Williams, R. M., Sanford, J. T., Stratford, P. W., et al. (1991) · Physical Therapy · doi:10.1093/ptj/71.9.679 · Cited by 11
Development and Validation of a Measure of Quality of Life for the Young Elderly in Sri Lanka
de Silva, S. H. P., Jayasuriya, A. R., Rajapaksa, L. C., Silva, A. P. D., & Barraclough, S. (2015) · Asia Pacific Journal of Public Health · doi:10.1177/1010539515625113 · Cited by 12
de Bakker, C. M. J., Altman, A. R., Li, C., et al. (2016) · Annals of Biomedical Engineering · doi:10.1007/s10439-015-1527-9 · Cited by 11
Russell, D. J., Ward, M., & Law, M. (1994) · The Occupational Therapy Journal of Research · doi:10.1177/153944929401400305 · Cited by 11
An Efficacy Trial of Carescapes: Home-Based Child-Care Practices and Children's Social Outcomes
Rusby, J. C., Jones, L. B., Crowley, R. N., & Smolkowski, K. (2016) · Child Development · doi:10.1111/cdev.12541 · Cited by 11
Park, Y.-J., Park, Y.-J., Lee, J.-M., et al. (2016) · 中医杂志：英文版 · doi:10.1016/s0254-6272(16)30023-1 · Cited by 11
Tanana, M., Dembe, A., Soma, C. S., Imel, Z. E., Atkins, D. C., & Srikumar, V. (2016) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w16-0304 · Cited by 11
Lee, S., Tang, S.-F., Lu, W.-S., Huang, S., Deng, N.-Y., Lue, W.-C., & Hsieh, C. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.10.058 · Cited by 11
Communicating a Key Benefit Claim Creatively and Effectively through Five Conveyor Properties
Althuizen, N. (2016) · Psychology and Marketing · doi:10.1002/mar.20970 · Cited by 11
Midwives perceptions of partner presence in childbirth pain alleviation in Nigeria hospitals
Emelonye, A. U., Vehviläinen-Julkunen, K. M., Pitkäaho, T., & Aregbesola, A. (2017) · Midwifery · doi:10.1016/j.midw.2017.03.004 · Cited by 11
McGee, C. W. (2017) · Journal of Hand Therapy · doi:10.1016/j.jht.2017.03.002 · Cited by 11
The contemptuous separation: Facial expressions of emotion and breakups in young adulthood
Heshmati, S., Sbarra, D. A., & Mason, A. E. (2017) · Personal Relationships · doi:10.1111/pere.12192 · Cited by 11
Oliveira, A., Cruz, J., Jácome, C., & Marques, A. (2017) · Physiotherapy Canada. Physiothérapie Canada · doi:10.3138/ptc.2016-42 · Cited by 10
Patient-Proxy Agreement on the Health-Related Quality of Life One Year After Traumatic Brain Injury
Hwang, H., Chen, C., & Lin, M. (2017) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2017.05.013 · Cited by 12
Standardized method for the measurement of Grabb’s line and clival-canal angle
Martin, J. E., Bookland, M. J., Moote, D. J., et al. (2017) · Journal of Neurosurgery Pediatrics · doi:10.3171/2017.5.peds17181 · Cited by 11
Systematic review of clinical practice guidelines in kidney transplantation
O’Donoghue, K., Reed, R. D., Knight, S. R., et al. (2017) · BJS Open · doi:10.1002/bjs5.17 · Cited by 11
Brown, R. C., Southam‐Gerow, M. A., McLeod, B. D., Wheat, E. B., Tully, C. B., Reise, S. P., Kendall, P. C., & Weisz, J. R. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22537 · Cited by 12
Mangone, M., Bernetti, A., Germanotta, M., et al. (2018) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2018.01.003 · Cited by 11
Dierick, F., Galtsova, E., Lauer, C., Buisseret, F., Bouché, A.-F., & Martin, L. (2018) · European Journal of Applied Physiology · doi:10.1007/s00421-018-3899-7 · Cited by 11
Wang, T., Wu, K. Y., Miner, R. C., Renaud, J. M., Beanlands, R. S., & deKemp, R. A. (2018) · EJNMMI Research · doi:10.1186/s13550-018-0421-5 · Cited by 11
Variability in the Echocardiographic Evaluation of the Systemic Right Ventricle
Chaix, M., Dore, A., Marcotte, F., et al. (2018) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2018.11.021 · Cited by 11
Interrater reliability of three-dimensional reconstruction of the spine
Almansour, H., Pepke, W., Rehm, J., et al. (2019) · Der Orthopäde · doi:10.1007/s00132-019-03712-x · Cited by 11
Reicherz, A., Brach, M., Cerny, J., et al. (2011) · Zeitschrift für Gerontologie und Geriatrie · doi:10.1007/s00391-011-0169-2 · Cited by 11
McAdams, L. A., Harris, M. J., Heaton, S. C., et al. (1997) · Schizophrenia Research · doi:10.1016/s0920-9964(97)00066-2 · Cited by 11
Júnior, E. A., Passos, A. P., Bruns, R. F., Nardozza, L. M. M., & Moron, A. F. (2013) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.3109/14767058.2013.847419 · Cited by 12
Global proteomic analysis distinguishes biologic differences in head and neck squamous carcinoma
Sudha, R., Kawachi, N., Du, P., et al. (2007) · Laboratory Investigation · doi:10.1038/labinvest.3700598 · Cited by 11
Čelar, A., Tamaki, K., Nitsche, S., & Schneider, B. (1999) · Journal of Prosthetic Dentistry · doi:10.1016/s0022-3913(99)70230-1 · Cited by 11
Haglund, L., Thorell, L., & Wålinder, J. (1998) · The Occupational Therapy Journal of Research · doi:10.1177/153944929801800405 · Cited by 11
Uehara, E. S., Srebnik, D., & Smukler, M. (2003) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1023/a:1024080532651 · Cited by 11
Evaluating the ALOCOM Approach for Scalable Content Repurposing
Verbert, K., & Duval, E. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-75195-3_26 · Cited by 11
Immunophenotyping of acute leukemia: Utility of CD45 for blast cell identification
Vial, J., & Lacombe, F. (2001) · Methods in cell biology · doi:10.1016/s0091-679x(01)64021-4 · Cited by 11
Reliability of bloodflux measurements from the upper trapezius muscle during muscle contractions
Røe, C., Damsgård, E., & Knardahl, S. (2007) · European Journal of Applied Physiology · doi:10.1007/s00421-007-0610-9 · Cited by 11
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
