The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 12,401–12,500 of 23,881
Kawaguchi, J. K., & Babcock, G. J. (2010) · Athletic Training & Sports Health Care · doi:10.3928/19425864-20101221-04 · Cited by 15
Test-retest variability of adenosine A2A binding in the human brain with 11C-TMSX and PET
Naganawa, M., Mishina, M., Sakata, M., Oda, K., Hiura, M., Ishii, K., & Ishiwata, K. (2014) · EJNMMI Research · doi:10.1186/s13550-014-0076-9 · Cited by 16
Sex, physical attractiveness, and perceived risk for mental illness.
O’Grady, K. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.43.5.1064 · Cited by 15
Genetic contributions to saliva protein concentrations in adult human twins
Rudney, J. D., Michalowicz, B. S., Krig, M. A., Kane, P. K., & Pihlstrom, B. L. (1994) · Archives of Oral Biology · doi:10.1016/0003-9969(94)90148-1 · Cited by 15
A new Stroke Activity Scale-results of a reliability study
Horgan, F., Finn, A. M., O'Regan, M. E., et al. (2003) · Disability and Rehabilitation · doi:10.1080/0963828021000031223 · Cited by 15
Dantas, C. D. R., & Banzato, C. E. M. (2007) · Brazilian Journal of Psychiatry · doi:10.1590/s1516-44462006005000041 · Cited by 15
Lorenz, J. M., Whitaker, A. H., Feldman, J. F., et al. (2009) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0b013e3181c35ee4 · Cited by 15
Philipse, M. W. G., Koeter, M. W. J., van der Staak, C. P. F., et al. (2005) · Criminal Justice and Behavior · doi:10.1177/0093854805279946 · Cited by 15
Intra‐session repeatability of FET and FEV6 in the general population
Kainu, A., Lindqvist, A. E., Sarna, S. J., & Sovijärvi, A. R. A. (2008) · Clinical Physiology and Functional Imaging · doi:10.1111/j.1475-097x.2008.00792.x · Cited by 15
Harrison, D. E., Harrison, D. E., Haas, J. W., et al. (2005) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2005.07.009 · Cited by 15
A prototype strategy for the construction of a creativity scale
Amelang, M., Herboth, G., & Oefner, I. (1991) · European Journal of Personality · doi:10.1002/per.2410050402 · Cited by 15
Criterion-based training for Rorschach scoring.
Hilsenroth, M. J., Charnas, J. W., Zodan, J., et al. (2007) · Training and Education in Professional Psychology · doi:10.1037/1931-3918.1.2.125 · Cited by 15
Morning Cortisol in Relation to Behavioral Symptoms of Nursing Home Residents With Dementia
Woods, D. L., Kim, H. T., & Yefimova, M. (2010) · Biological Research For Nursing · doi:10.1177/1099800410385568 · Cited by 16
Bernardo, M. V., Crawford, P. F., Hertel, J., Sholer, C., Xu, X., Gross, T. J., Kewalramani, R., & GLOBE, D. R. (2006) · Journal of Managed Care Pharmacy · doi:10.18553/jmcp.2006.12.9.714 · Cited by 15
Kanter, J. W., Rusch, L. C., Landes, S. J., Holman, G. I., Whiteside, U. S., & Sedivy, S. K. (2009) · Psychotherapy · doi:10.1037/a0016083 · Cited by 15
Do Raters Agree More on Observable Items?
Roch, S. G., Paquin, A. R., & Littlejohn, T. W. (2009) · Human Performance · doi:10.1080/08959280903248344 · Cited by 15
RELIABILITY AND VALIDITY OF A DEVICE TO MEASURE MUSCLE HARDNESS
Kato, G., Andrew, P. D., & Sato, H. (2004) · Journal of Mechanics in Medicine and Biology · doi:10.1142/s0219519404000989 · Cited by 15
Kamans, E., Otten, S., Gordijn, E., & Spears, R. (2010) · Group Processes & Intergroup Relations · doi:10.1177/1368430210375252 · Cited by 15
The Reliability of the Vestibular Autorotation Test (VAT) in Patients with Dizziness
Blatt, P. J., Schubert, M. C., Roach, K. E., et al. (2008) · Neurology Report · doi:10.1097/npt.0b013e3181733709 · Cited by 15
Becker, A. E., Fay, K., Agnew‐Blais, J. C., Guarnaccia, P. J., Striegel‐Moore, R. H., & Gilman, S. E. (2010) · Transcultural Psychiatry · doi:10.1177/1363461510382153 · Cited by 15
Potdevin, F. J., Femery, V. G., Decatoire, A., et al. (2007) · The Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2006.07.0077 · Cited by 15
Hardesty, C. K., Aronson, J., Aronson, E. A., et al. (2013) · Journal of Pediatric Orthopaedics · doi:10.1097/bpo.0b013e3182770bd3 · Cited by 15
Zügner, R., Tranberg, R., Lisovskaja, V., et al. (2016) · Journal of Orthopaedic Research® · doi:10.1002/jor.23415 · Cited by 15
Anota, A., Mariet, A., Maingon, P., Joly, F., Bosset, J.-F., Guizard, A., Bittard, H., Velten, M., & Mercier, M. (2016) · Health and Quality of Life Outcomes · doi:10.1186/s12955-016-0571-y · Cited by 14
A simple method to measure gait for use in arthritis clinical research
Falconer, J., & Hayes, P. W. (1991) · Arthritis & Rheumatism · doi:10.1002/art.1790040110 · Cited by 15
The Effect of Mobilization on Forward Bending Range
Gibson, H. M., Ross, J., Allen, J., et al. (1993) · Journal of Manual & Manipulative Therapy · doi:10.1179/jmt.1993.1.4.142 · Cited by 15
The Reliability and Validity of Hand-Held Refractometry Water Content Measures of Hydrogel Lenses
Nichols, J. J., Mitchell, G. L., & GOOD, A. G. W. (2003) · Optometry and Vision Science · doi:10.1002/j.1538-9235.2003.tb01669.x · Cited by 15
Effect of a Feldenkrais Intervention on Symptomatic Subjects Performing a Functional Reach
Chinn, J., Trujillo, D. J., Kegerreis, S. T., et al. (1994) · Isokinetics and Exercise Science · doi:10.3233/ies-1994-4401 · Cited by 15
Evidence of observer reliability for the Fagan Test of Infant Intelligence (FTII)
O'Neill, J. M., Jacobson, S. W., & Jacobson, J. L. (1994) · Infant Behavior and Development · doi:10.1016/0163-6383(94)90039-6 · Cited by 15
A Coming‐of‐Age for Research on Interpersonal Perception
Kenny, D. A. (1993) · Journal of Personality · doi:10.1111/j.1467-6494.1993.tb00791.x · Cited by 15
Soldz, S., Budman, S. H., Demby, A., & Merry, J. (1995) · Psychotherapy · doi:10.1037/0033-3204.32.4.678 · Cited by 15
Congruence of depressive symptom appraisal between persons with stroke and their caregivers.
Klinedinst, N. J., Clark, P. C., Blanton, S., & Wolf, S. L. (2007) · Rehabilitation Psychology · doi:10.1037/0090-5550.52.2.215 · Cited by 15
Letrait, M., Lurie, A., Bean, K., Mesbah, M., Alain Venot, A., Strauch, G., Grandordy, B., & Chwalow, J. (1996) · European Respiratory Journal · doi:10.1183/09031936.96.09061167 · Cited by 15
Reproducibility of manual pressure force on provocation of the sacroiliac joint
Levin, U., Nilsson-Wikmar, L. B., Stenstruöm, C. H., et al. (1998) · Physiotherapy Research International · doi:10.1002/pri.118 · Cited by 15
Hooper visual organization test: Effects of object naming ability
Ricker, J. H., & Axelrod, B. N. (1995) · The Clinical Neuropsychologist · doi:10.1080/13854049508402058 · Cited by 15
Statistical Methods for Assessing Biomarkers
Looney, S. W. (2003) · Humana Press eBooks · doi:10.1385/1-59259-242-2:081 · Cited by 15
Reliability of the Contour Test in a population of adults with hearing loss.
Cv, P., & th, L. G. (1998) · PubMed · Cited by 15
[Comparative analysis of three approaches for rater agreement].
Ato, M., Benavente, A., & García, J. J. L. (2006) · PubMed · Cited by 15
Mortifee, P. R. S., Busser, J. R., & Anton, H. A. (1996) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(96)90030-1 · Cited by 15
Predicting ESL learners’ oral proficiency by measuring the collocations in their spontaneous speech
Xu, J. (2015) · Iowa State University Digital Repository (Iowa State University) · doi:10.31274/etd-180810-4474 · Cited by 17
Luchini, S. A., Maliakkal, N. T., DiStefano, P. V., Patterson, J. D., Beaty, R. E., & Reiter‐Palmon, R. (2023) · doi:10.31234/osf.io/g5qvf · Cited by 15
Distributed Learning in the Classroom: Effects of Rereading Schedules Depend on Time of Test
Greving, C. E., & Richter, T. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02517 · Cited by 15
Mott, J. M., Barrera, T. L., Hernandez, C., Graham, D. P., & Teng, E. J. (2014) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-013-9352-0 · Cited by 14
Czaprowski, D., Kędra, A., Pawłowska, P., Kolwicz-Gańko, A., Leszczewska, J., & Tyrakowski, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0121360 · Cited by 14
Sharma, S., Kallen, M. A., & Ohrbach, R. K. (2021) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000001005 · Cited by 35
Erkkilä, M., Peräkylä, J., & Hartikainen, K. M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02556 · Cited by 20
A Psychometric Validation of the PAILQ-6: Perceived Artificial Intelligence Literacy Questionnaire
Grassini, S. (2024) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3679318.3685359 · Cited by 23
Psychometric Properties and Meaningful Change Thresholds of the Vitiligo Area Scoring Index
Ezzedine, K., Soliman, A. M., Camp, H. S., et al. (2024) · JAMA Dermatology · doi:10.1001/jamadermatol.2024.4534 · Cited by 14
Bridging the gap to clinical use: A systematic review on TMS-EEG test-retest reliability.
Bertazzoli, G., Dognini, E., Fried, P. J., Miniussi, C., Julkunen, P., & Bortoletto, M. (2024) · Clinical Neurophysiology · doi:10.1016/j.clinph.2025.01.002 · Cited by 14
Acosta-Enriquez, B. G., Huamaní-Jordan, O., Morales-Angaspilco, J. E., Heredia-Pérez, O., Ruiz-Carrillo, J. R., Blanco-García, L. E., & de Villalobos, S. M. V. P. (2025) · BMC Psychology · doi:10.1186/s40359-025-03367-8 · Cited by 16
Park, I.-W., Lim, O.-B., Park, K.-N., et al. (2014) · Physical Therapy Korea · doi:10.12674/ptk.2014.21.1.001 · Cited by 14
Ericsson, A., Nyström, C., & Mannerkorpi, K. (2014) · Nordic Journal of Psychiatry · doi:10.3109/08039488.2014.915580 · Cited by 14
Dusting Off the Messy Middle: Assessing Students’ Inquiry Skills Through Doing and Writing
Li, H., Gobert, J. D., & Dickler, R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-61425-0_15 · Cited by 14
Ranjan, M., Boutet, A., Xu, D. S., et al. (2018) · Stereotactic and Functional Neurosurgery · doi:10.1159/000488397 · Cited by 14
Wang, K. K., Stout, J. L., Ries, A. J., et al. (2018) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.14051 · Cited by 14
Eye-movement efficiency and sight-reading expertise in woodwind players
Zhukov, K., Khuu, S. K., & McPherson, G. E. (2019) · Journal of Eye Movement Research · doi:10.16910/jemr.12.2.6 · Cited by 14
Cultural equity: knowledge and outcomes aspects
Torelli, C. J., Oh, H., & Stoner, J. L. (2020) · International Marketing Review · doi:10.1108/imr-12-2018-0366 · Cited by 14
Measuring Readiness for Kindergarten Using the Desired Results Developmental Profile
Kriener-Althen, K., Newton, E. K., Draney, K., et al. (2020) · Early Education and Development · doi:10.1080/10409289.2020.1743160 · Cited by 14
Agreement Among Paper and Electronic Modes of the EQ-5D-5L
Lundy, J. J., Coons, S. J., Flood, E. M., & Patel, M. J. (2020) · Patient · doi:10.1007/s40271-020-00419-6 · Cited by 14
Aberegg, E., Collins, K. K., Hinderliter, J. M., Smalling, S. L., Fung, T. T., Gizlice, Z., Johnston, L. F., Ammerman, A. S., & Keyserling, T. C. (2020) · Journal of Cardiopulmonary Rehabilitation and Prevention · doi:10.1097/hcr.0000000000000505 · Cited by 14
KANITA DAYALI UYGULAMA ÖLÇEĞİNİN TÜRKÇEYE UYARLANMASI: GEÇERLİK VE GÜVENİRLİK ÇALIŞMASI
ÇAY, M., & Daşbaş, S. (2020) · Toplum ve sosyal hizmet(Online)/Toplum ve sosyal hizmet · doi:10.33417/tsh.733462 · Cited by 14
Quantifying the Degree of Bruise Visibility Observed Under White Light and an Alternate Light Source
Scafide, K. N., Downing, N. R., Kutahyalioglu, N. S., Sebeh, Y., Sheridan, D. J., & Hayat, M. J. (2020) · Journal of Forensic Nursing · doi:10.1097/jfn.0000000000000304 · Cited by 14
Valério, C. S., Cardoso, C. A. E. A., Araújo, E., et al. (2021) · Imaging Science in Dentistry · doi:10.5624/isd.20200260 · Cited by 14
Wu, S., Nijstad, B. A., & Yuan, Y. (2021) · Group Processes & Intergroup Relations · doi:10.1177/1368430221999457 · Cited by 14
Road verge extent and habitat composition across Great Britain
Phillips, B. B., Navaratnam, A., Hooper, J., et al. (2021) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2021.104159 · Cited by 14
Gagnon, M., Merlo, G. M., Yap, R., Collins, J., Elfassy, C., Sawatzky, B., Marsh, J. D., Hamdy, R. C., Veilleux, L., & Dahan‐Oliel, N. (2021) · Journal of Medical Internet Research · doi:10.2196/27064 · Cited by 14
Fraser, M. A., Walsh, E., Shaw, M. E., Anstey, K. J., & Cherbuin, N. (2021) · Cerebral Cortex · doi:10.1093/cercor/bhab375 · Cited by 15
Boucher, J., Brisebois, A., Slegers, A., Courson, M., Désilets-Barnabé, M., Chouinard, A.-M., Gbeglo, V., Marcotte, K., & Brambati, S. M. (2021) · American Journal of Speech-Language Pathology · doi:10.1044/2021_ajslp-20-00388 · Cited by 14
Ranmuthu, C. D. S., Mackay, J., Crowe, V. A., Kaggie, J. D., Kessler, D. A., & McDonnell, S. (2021) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-021-04755-y · Cited by 14
Yu, H., Tong, T., Gao, Y., et al. (2021) · BMC Nursing · doi:10.1186/s12912-021-00768-z · Cited by 14
Regional Well-Being and its Inequality in the OECD Member Countries
Liberati, P., & Resce, G. (2022) · The Journal of Economic Inequality · doi:10.1007/s10888-021-09521-7 · Cited by 14
Sánchez, D. G., Pajares-Hachero, B., Trinidad‐Fernández, M., Escriche-Escuder, A., Campos, M. I., Bermejo-Pérez, M. J., Alba, E., Roldán‐Jiménez, C., & Cuesta‐Vargas, A. (2022) · Pain Management Nursing · doi:10.1016/j.pmn.2022.01.003 · Cited by 15
Fiala, V., Tureček, P., Akoko, R. M., Pokorný, Š., & Kleisner, K. (2022) · Scientific Reports · doi:10.1038/s41598-022-10646-6 · Cited by 15
Reliability of 3D planning and simulations of medial open wedge high tibial osteotomies
Jörgens, M., Keppler, A. M., Degen, N., et al. (2022) · Journal of orthopaedic surgery · doi:10.1177/10225536221101699 · Cited by 14
Hirschmüller, S., Mosel, E. I., & Egloff, B. (2022) · Journal of Personality · doi:10.1111/jopy.12756 · Cited by 14
Jung, L., Rufener, C., & Petow, S. (2022) · Frontiers in Veterinary Science · doi:10.3389/fvets.2022.937119 · Cited by 14
Yoon, L. S., Binder, A. M., Pereira, A. P. R., Calafat, A. M., Shepherd, J., Corvalán, C., & Michels, K. B. (2022) · Environment International · doi:10.1016/j.envint.2022.107586 · Cited by 15
Comer, A. R., Templeton, E., Glidden, M., et al. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2022.1093392 · Cited by 14
Maskeliūnas, R., Kulikajevas, A., Damaševičius, R., Griškevičius, J., & Adomavičienė, A. (2023) · Applied Sciences · doi:10.3390/app13021116 · Cited by 18
Violence, neurocognitive function and clinical correlates in patients with schizophrenia
Yi, Y., Huang, Y., Chen, Q., Yang, H., Li, H., Feng, Y., Feng, S., Zhou, S., Li, Z., & Wu, F. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.1087372 · Cited by 14
Beck, F., Lettner, S., Cepic, L. Z., et al. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12030996 · Cited by 14
The Acute Effect of High-Intensity Resistance Training on Subsequent Firefighter Performance
Mason, M. R., Heebner, N. R., Abt, J. P., Bergstrom, H. C., Shapiro, R., Langford, E. L., & Abel, M. G. (2023) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000004417 · Cited by 14
Minimum Detectable Differences in Electrocochleography Measurements: Bayesian-Based Predictions
Goodman, S. S., Lichtenhan, J. T., & Jennings, S. G. (2023) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-023-00888-0 · Cited by 14
Performance during presentations: A question of challenge and threat responses?
Schickel, M., Minkley, N., & Ringeisen, T. (2023) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102168 · Cited by 14
Correlation of age and skeletal effects after miniscrew assisted rapid palatal expansion
Marin, C., Benitez, M., Otazu, A., et al. (2023) · Journal of Clinical and Experimental Dentistry · doi:10.4317/jced.60211 · Cited by 14
Shahidullah, J. D., Custer, J., Widales-Benitez, O., Aksan, N. S., Hatchell, C., Newport, D. J., Wagner, K. D., Storch, E. A., Claassen, C. A., Garrett, A. S., et al. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115168 · Cited by 15
Thompson, B. J., & Xu, J. (2023) · Journal of Functional Morphology and Kinesiology · doi:10.3390/jfmk8020062 · Cited by 14
Italian validation of the Pelvic Floor Distress Inventory (PFDI-20) questionnaire
Barba, M., Cola, A., Melocchi, T., Braga, A., Castronovo, F., Manodoro, S., Pennacchio, M., Munno, G. M., Ruffolo, A. F., Degliuomini, R. S., et al. (2023) · International Urogynecology Journal · doi:10.1007/s00192-023-05572-8 · Cited by 14
Resnick, P., Alfayez, A., Im, J., & Gilbert, É. (2023) · Collective Intelligence · doi:10.1177/26339137231173407 · Cited by 14
Cosío, B. G., Shafiek, H., Iglesias, A., et al. (2023) · Archivos de Bronconeumología · doi:10.1016/j.arbres.2023.05.014 · Cited by 14
Yamamori, Y., Robinson, O. J., & Roiser, J. P. (2023) · eLife · doi:10.7554/elife.87720 · Cited by 26
Ranjbar, S., Mazidi, M., Gross, J. J., Preece, D. A., Zarei, M., Azizi, A., Mirshafiei, M., & Becerra, R. (2023) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-023-10068-2 · Cited by 14
Oosterwegel, M. J., Ibi, D., Portengen, L., Probst-Hensch, N. M., Tarallo, S., Naccarati, A., Imboden, M., Jeong, A., Robinot, N. R., Scalbert, A., et al. (2023) · Environmental Science & Technology · doi:10.1021/acs.est.3c03233 · Cited by 16
Stipancic, K. L., Golzy, M., Zhao, Y., et al. (2023) · Journal of Speech Language and Hearing Research · doi:10.1044/2023_jslhr-23-00322 · Cited by 14
Test-retest reliability of joint position sense in the elbow among healthy adults
Rider, J. V., & Valdes, K. A. (2023) · Journal of Hand Therapy · doi:10.1016/j.jht.2023.08.015 · Cited by 15
Loebach, J. E., Ramsden, R., Cox, A., Joyce, K., & BRUSSONI, M. J. (2023) · Journal of Outdoor and Environmental Education · doi:10.1007/s42322-023-00145-1 · Cited by 15
Teixeira, R. C. P., & Cahali, M. B. (2024) · Sensors · doi:10.3390/s24092803 · Cited by 14
Luque, L., Skogen, K., MacIntosh, B. J., et al. (2024) · Frontiers in Radiology · doi:10.3389/fradi.2024.1357341 · Cited by 14
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
