The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 11,001–11,100 of 23,881
Gullekson, N. L., & Vancouver, J. B. (2009) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2009.05.002 · Cited by 19
Hamilton, D. A., Mahoney, M. C., Novalen, M., et al. (2015) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntv031 · Cited by 19
Rusby, J. C., Jones, L. B., Crowley, R. N., & Smolkowski, K. (2013) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2013.02.003 · Cited by 19
Ikuma, L. H., Nussbaum, M. A., & Babski-Reeves, K. L. (2009) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2009.02.005 · Cited by 19
Verma, S. K., McClure, K., Parker, L., et al. (2010) · Clinical Radiology · doi:10.1016/j.crad.2009.09.016 · Cited by 19
Harvey, L. A., King, M., & Herbert, R. (1994) · Journal of Hand Therapy · doi:10.1016/s0894-1130(12)80244-4 · Cited by 19
Baxter, J. A., Barlow, T., Karthikeyan, S., et al. (2012) · Hip International · doi:10.5301/hip.2012.9044 · Cited by 19
Preliminary studies on the effect of moderate physical activity on blood levels of glutathione
Rundle, A., Orjuela, M. A., Mooney, L. A., Tang, D. L., Kim, M. J., Calcagnotto, A., Richie, J. P., & Perera, F. P. (2005) · Biomarkers · doi:10.1080/13547500500272663 · Cited by 19
Moderators of the Effects of Treatment Length in Long-Term Psychodynamic Group Psychotherapy
Lorentzen, S., & Høglend, P. A. (2008) · Psychotherapy and Psychosomatics · doi:10.1159/000147946 · Cited by 19
The reliability of measuring quadriceps cross‐sectional area with compound B ultrasound scanning
Howe, T. E., & Oldham, J. A. (1996) · Physiotherapy Research International · doi:10.1002/pri.6120010207 · Cited by 19
Automated 3D Segmentation of Intraretinal Surfaces in SD-OCT Volumes in Normal and Diabetic Mice
Antony, B. J., Jeong, W., Abràmoff, M. D., et al. (2014) · Translational Vision Science & Technology · doi:10.1167/tvst.3.5.8 · Cited by 19
The School Mental Health Capacity Instrument: Development of an Assessment and Consultation Tool
Feigenberg, L. F., Watts, C. L., & Buckner, J. C. (2010) · School Mental Health · doi:10.1007/s12310-010-9041-6 · Cited by 21
Rouse, P. C., Duda, J. L., Ntoumanis, N., Jolly, K. B., & Williams, G. C. (2014) · European Journal of Sport Science · doi:10.1080/17461391.2014.987320 · Cited by 21
Reproducibility of Tract-based and Region-of-Interest DTI Analysis of Long Association Tracts
Brandstack, N., Kurki, T. J. I., Laalo, J. P., Kauko, T., & Tenovuo, O. S. (2014) · Clinical Neuroradiology · doi:10.1007/s00062-014-0349-8 · Cited by 19
Defence and coping in bipolar affective disorder: Stability and change of adaptational processes
Kramer, U. (2009) · British Journal of Clinical Psychology · doi:10.1348/014466509x457397 · Cited by 19
Klintberg, I. H., LIND, K., Marlow, T., & Svantesson, U. M. (2012) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2011.11.027 · Cited by 18
Nelson, B. D., Shankman, S. A., Olino, T. M., & Klein, D. N. (2011) · Cognition & Emotion · doi:10.1080/02699931.2010.551185 · Cited by 21
EMG-Torque Relationship and Reliability of the Medial and Lateral Hamstring Muscles
Campy, R. M., COELHO, A. J., & Pincivero, D. M. (2009) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3181a8c4cb · Cited by 19
Rosar, U., & Klein, M. (2009) · KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie · doi:10.1007/s11577-009-0086-1 · Cited by 19
Association between stereotypic behavior and polydipsia in chronic schizophrenic patients
Shutty, M. S., Mcculley, K., & Pigott, B. (1995) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(95)00049-6 · Cited by 19
Hakki's Formula for Measurement of Aortic Valve Area by Magnetic Resonance Imaging
Puymirat, É., Chassaing, S., Trinquart, L., Barbey, C., Chaudeurge, A., Bar, O., & Blanchard, D. (2010) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2010.03.019 · Cited by 19
Differences in Inter‐Rater Reliability and Accuracy for a Treatment Adherence Scale
Wu, S. M., Whiteside, U. S., & Neighbors, C. (2007) · Cognitive Behaviour Therapy · doi:10.1080/16506070701584367 · Cited by 19
Digital Imaging of Wounds: Are Measurements Reproducible Among Observers?
Quan, S. Y., Lazarus, G. S., Kohli, A. R., et al. (2007) · The International Journal of Lower Extremity Wounds · doi:10.1177/1534734607306880 · Cited by 19
Pubic Symphyseal Width in Pediatric Patients
McAlister, D. M., Webb, H. R., Wheeler, P. D., et al. (2005) · Journal of Pediatric Orthopaedics · doi:10.1097/01.bpo.0000167080.91459.87 · Cited by 19
Effects of music therapy on facial expression of individuals with Parkinson’s disease: A pilot study
Elefant, C., Lotan, M., Baker, F. A., & SKEIE, G. O. (2012) · Musicae Scientiae · doi:10.1177/1029864912458917 · Cited by 19
System functioning in families with ADHD and non‐ADHD sons
Camparo, L. B., Christensen, A., Buhrmester, D., et al. (1994) · Personal Relationships · doi:10.1111/j.1475-6811.1994.tb00067.x · Cited by 19
Czamara, A., Markowska, I., Królikowska, A., et al. (2015) · BioMed Research International · doi:10.1155/2015/707168 · Cited by 19
Koerte, I. K., Haberl, C., Schmidt, M., et al. (2013) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.24013 · Cited by 19
Neil‐Sztramko, S., Myring, A., Peeters, M. J., Pirie, I., Jacobs, R., Hunt, M. A., Garland, S. J., & Campbell, K. L. (2013) · Physiotherapy Theory and Practice · doi:10.3109/09593985.2013.779337 · Cited by 20
Body weight, social competence, and cognitive functioning in survivors of childhood brain tumors
Schulte, F., Bartels, U. K., Bouffet, É., et al. (2010) · Pediatric Blood & Cancer · doi:10.1002/pbc.22543 · Cited by 19
Familial Liability to Schizophrenia: A Sibling Study of Negative Symptoms
Craver, J. C., & Pogue‐Geile, M. F. (1999) · Schizophrenia Bulletin · doi:10.1093/oxfordjournals.schbul.a033422 · Cited by 19
Pérez, S. I., Bernal, V., & González, P. (2007) · Human Biology · doi:10.1353/hub.2007.0027 · Cited by 19
Cummings, J. A., Hayes, A. M., Cardaciotto, L., & Newman, C. F. (2011) · Cognitive Therapy and Research · doi:10.1007/s10608-011-9375-x · Cited by 23
Effect of a repeated jaw motor task on masseter muscle performance
Iida, T., Komiyama, O., Honki, H., et al. (2015) · Archives of Oral Biology · doi:10.1016/j.archoralbio.2015.08.005 · Cited by 19
Meeting facilitation: process versus content interventions
Miranda, S. M., & Bostrom, R. P. (2002) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.1997.665468 · Cited by 19
Reliability of a digital image method for measuring medial mandibular flexure in dentate subjects
Shinkai, R. S. A., Canabarro, S. D. A., Schmidt, C. B., & Sartori, E. A. (2004) · Journal of Applied Oral Science · doi:10.1590/s1678-77572004000400020 · Cited by 19
Wöpking, S., Scherens, A., Haussleiter, I. S., Richter, H., Schüning, J., Klauenberg, S., & Maier, C. (2009) · BMC Neurology · doi:10.1186/1471-2377-9-13 · Cited by 19
Bullock, S. L., Craypo, L., Clark, S. E., et al. (2010) · Journal of the American Dietetic Association · doi:10.1016/j.jada.2010.04.002 · Cited by 19
Clinical Peer Review Program Self-Evaluation for US Hospitals
Edwards, M. T. (2010) · American Journal of Medical Quality · doi:10.1177/1062860610371224 · Cited by 19
Cahill, N. E., Day, A. G., Cook, D., & Heyland, D. K. (2013) · Implementation Science · doi:10.1186/1748-5908-8-140 · Cited by 19
Lots of target variance: An update of SRM using the HEXACO personality inventory
de Vries, R. E. (2010) · European Journal of Personality · doi:10.1002/per.764 · Cited by 20
Test-retest reliability of fMRI experiments during robot-assisted active and passive stepping
Jaeger, L., Marchal–Crespo, L., Wolf, P., Riener, R., Kollias, S., & Michels, L. (2015) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-015-0097-2 · Cited by 19
Effects of Energy-Matched Pulsed and Continuous Ultrasound on Tumor Growth in Mice
Sicard-Rosenbaum, L., Danoff, J., Guthrie, J. A., et al. (1998) · Physical Therapy · doi:10.1093/ptj/78.3.271 · Cited by 19
Siemionow, K. B., Monsef, J. B., & Janusz, P. (2016) · World Neurosurgery · doi:10.1016/j.wneu.2016.01.053 · Cited by 19
Goodman, G., Anderson, K., & Diener, M. J. (2014) · Journal of Psychotherapy Integration · doi:10.1037/a0035970 · Cited by 19
A novel ultrasound technique to measure genioglossus movement in vivo
Kwan, B. C. H., Butler, J. E., Hudson, A. L., McKenzie, D. K., Bilston, L. E., & Gandevia, S. C. (2014) · Journal of Applied Physiology · doi:10.1152/japplphysiol.01257.2013 · Cited by 19
Zhang, L., Jiang, N., Li, Z., et al. (2016) · Cancer Nursing · doi:10.1097/ncc.0000000000000397 · Cited by 19
Situation Characteristics Are Age Graded
Brown, N. A., & Rauthmann, J. F. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550616652207 · Cited by 19
Harel, D., Hitchcock, E. R., Szeredi, D., Ortiz, J. A., & Byun, T. M. (2016) · Clinical Linguistics & Phonetics · doi:10.3109/02699206.2016.1174306 · Cited by 19
Freire-Maia, B. A. V., Machado, V. D., Valério, C. S., et al. (2016) · International Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.ijom.2016.11.007 · Cited by 19
Bley, S. (2017) · Empirical research in vocational education and training · doi:10.1186/s40461-017-0049-0 · Cited by 19
Töger, J., Sorensen, T., Somandepalli, K., Toutios, A., Lingala, S. G., Narayanan, S. S., & Nayak, K. S. (2017) · The Journal of the Acoustical Society of America · doi:10.1121/1.4983081 · Cited by 18
Peña-Araya, V., Quezada, M., Poblete, B., et al. (2017) · EPJ Data Science · doi:10.1140/epjds/s13688-017-0122-8 · Cited by 19
Graph theory reveals amygdala modules consistent with its anatomical subdivisions
Caparelli, E. C., Ross, T. J., Gu, H., Liang, X., Stein, E. A., & Yang, Y. (2017) · Scientific Reports · doi:10.1038/s41598-017-14613-4 · Cited by 22
Moncrief, T., Gor, R. A., Goldfarb, R. A., et al. (2017) · The Journal of Urology · doi:10.1016/j.juro.2017.11.110 · Cited by 19
Identification of Ribosome Pause Sites Using a Z-Score Based Peak Detection Algorithm
Perkins, P., & Heber, S. (2018) · IEEE International Conference on Computational Advances in Bio and Medical Sciences · doi:10.1109/iccabs.2018.8541902 · Cited by 19
McDowell, Y. E., Verges, A. J., & Sher, K. J. (2019) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.13952 · Cited by 19
Okuda, M., Asakura, K., & Sasaki, S. (2019) · Nutrients · doi:10.3390/nu11020319 · Cited by 20
Global Emergency Medicine: A Review of the Literature From 2013
Becker, T. K., Jacquet, G. A., Marsh, R. H., Schroeder, E. D., Foran, M. P., Bartels, S. A., Duber, H. C., Cockrell, H. C., Levine, A. C., & the Global Emergency Medicine Literature Review (GEMLR) Group (2014) · Academic Emergency Medicine · doi:10.1111/acem.12414 · Cited by 19
Infants’ behavioral styles in joint attention situations and parents’ socio-economic status
Abels, M., & Hutman, T. (2015) · Infant Behavior and Development · doi:10.1016/j.infbeh.2015.05.004 · Cited by 23
Automated Assessment of Oral Reading Prosody
Mostow, J., & Duong, M. (2009) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-60750-028-5-189 · Cited by 19
Johansson, M., Carlberg, E. B., & Jylli, L. (2010) · Acta Paediatrica · doi:10.1111/j.1651-2227.2009.01632.x · Cited by 21
Effect of preload and range of motion on isokinetic torque in women
Tis, L. L., Perrin, D. H., WELTMAN, A. L., et al. (1993) · Medicine & Science in Sports & Exercise · doi:10.1249/00005768-199309000-00011 · Cited by 19
Effect of patient position on the consistency of placing the rearfoot at subtalar neutral
Pierrynowski, M. R., & Smith, S. (1997) · Journal of the American Podiatric Medical Association · doi:10.7547/87507315-87-9-399 · Cited by 19
Validity of Axis V of DSM‐III and other measures of adaptive functioning
Rey, J. M., Stewart, G. W., Plapp, J. M., et al. (1988) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1988.tb05166.x · Cited by 19
Measuring psychological distress symptoms in individuals living with HIV in western Kenya
Shacham, E., Reece, M., Monahan, P. O., Yebei, V. N., Omollo, O., Ong’or, W. O., & Ojwang, C. (2008) · Journal of Mental Health · doi:10.1080/09638230701530192 · Cited by 19
Outcomes after head injury: level of agreement between subjects and their informants
Ocampo, S., DAWSON, D. R., & Colantonio, A. (1997) · Occupational Therapy International · doi:10.1002/oti.54 · Cited by 19
Wong, S. P., & McGraw, K. O. (1999) · Educational and Psychological Measurement · doi:10.1177/00131649921969848 · Cited by 19
Tavares, L. D. A., Neto, J. B., Jardim, J. R., Souza, G. M. D. C. E., Hlatky, M. A., & Nascimento, O. A. (2012) · Jornal Brasileiro de Pneumologia · doi:10.1590/s1806-37132012000600002 · Cited by 19
Srinivasan, M., Litzelman, D. K., SESHADRI, R. M., et al. (2004) · Academic Medicine · doi:10.1097/00001888-200409000-00017 · Cited by 19
Measurement of teacher performance: A study in instrument development
Peterson, D., Micceri, T., & Smith, B. O. (1985) · Teaching and Teacher Education · doi:10.1016/0742-051x(85)90030-7 · Cited by 19
Degree of psychopathy—implications for treatment in male juvenile delinquents
Dåderman, A. M., & Kristiansson, M. (2003) · International Journal of Law and Psychiatry · doi:10.1016/s0160-2527(03)00040-2 · Cited by 19
Personality traits predicting long‐term adjustment after surgery for ulcerative colitis
Weinryb, R. M., Gustavsson, P. J., & Barber, J. P. (2003) · Journal of Clinical Psychology · doi:10.1002/jclp.10191 · Cited by 19
Stiles, W. B., Shankland, M. C., Wright, J., & Field, S. D. (1997) · Psychotherapy Research · doi:10.1080/10503309712331331943 · Cited by 19
Coster, W. J., Tickle‐Degnen, L., & Armenta, L. (1995) · The Occupational Therapy Journal of Research · doi:10.1177/153944929501500102 · Cited by 19
Quantification of microvessels in canine lymph nodes
Tonar, Z., Egger, G. F., Witter, K., & Wolfesberger, B. (2008) · Microscopy Research and Technique · doi:10.1002/jemt.20619 · Cited by 19
Långström, N. (2000) · Nordic Journal of Psychiatry · doi:10.1080/080394800427401 · Cited by 19
A two-process theory of facial perception of emotions
de Bonis, M., de Boeck, P. A. L., Pérez-Díaz, F., & Lévy-Nahas, M. (1999) · Comptes Rendus de l Académie des Sciences - Series III - Sciences de la Vie · doi:10.1016/s0764-4469(99)80106-1 · Cited by 19
Measuring Restrictiveness of Psychiatric Care
Ransohoff, P. M., Zachary, R. A., Gaynor, J., et al. (1982) · Psychiatric Services · doi:10.1176/ps.33.5.361 · Cited by 19
Cueing for transfer in multimedia programmes: process worksheets vs. worked‐out examples
Hummel, H. G. K., PAAS, F. G. W. C., & Koper, E. J. R. (2004) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2004.00098.x · Cited by 19
Venkatesh, G., Fallen, E. L., Kamath, M. V., et al. (1996) · Heart · doi:10.1136/hrt.76.2.137 · Cited by 19
Fellrath, J. (2007) · Swiss Medical Weekly · doi:10.4414/smw.2007.11741 · Cited by 19
The Karolinska Psychodynamic Profile (KAPP): Studies of character and well-being.
Weinryb, R. M., Rössel, R. J., Gustavsson, P. J., et al. (1997) · Psychoanalytic Psychology · doi:10.1037/h0079737 · Cited by 19
Basco, M. R., Birchler, G. R., Kalal, B., Talbott, R. E., & Slater, M. A. (1991) · Journal of Clinical Psychology · doi:10.1002/1097-4679(199105)47:3 · Cited by 19
Martinsen, E. W., Friis, S., & Hoflart, A. (1989) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1989.tb03011.x · Cited by 19
Reliability of Cybex II isokinetic evaluations of torque in post-poliomyelitis syndrome
Kilfoil, M. R., & ST.-PIERRE, D. M. M. (1993) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/0003-9993(93)90034-8 · Cited by 19
O'Brien, J. T., Ames, D. J., & SCHWIETZER, I. (1996) · International Journal of Geriatric Psychiatry · doi:10.1002/(sici)1099-1166(199608)11:8 · Cited by 73
Concordance between Patients and Their Case Managers Using the Health of the Nation Outcome Scales
Trauer, T., & Callaly, T. (2002) · Australasian Psychiatry · doi:10.1046/j.1440-1665.2002.00387.x · Cited by 19
Comparative study on the ALA photodynamic effects of human glioma and meningioma cells
Tsai, J., Hsiao, Y., Teng, L., Chen, C., & Kao, M. (1999) · Lasers in Surgery and Medicine · doi:10.1002/(sici)1096-9101(1999)24:4 · Cited by 52
Tohka, J., Wallius, E., Hirvonen, J., et al. (2006) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2005.862971 · Cited by 19
Low blood pressure associated with low mood: A red herring?
Donner‐Banzhoff, N., Chan, Y., Szalai, J. P., et al. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00157-1 · Cited by 19
Validating Specific Psychopathology Scales in Older Outpatients with Schizophrenia
McAdams, L. A., Harris, M. J., Bailey, A. L., Fell, R. L., & Jeste, D. V. (1996) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199604000-00008 · Cited by 19
Sharma, T., du Boulay, G. H., Lewis, S. W., Sigmundsson, T., Gurling, H., & Murray, R. M. (1997) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/s0278-5846(97)00165-6 · Cited by 19
Hussey, I., & Drake, C. E. (2020) · doi:10.31234/osf.io/ge3k7 · Cited by 20
Rathschlag, M., & Memmert, D. (2014) · Brain and Behavior · doi:10.1002/brb3.221 · Cited by 18
Gasteiger, N., Vercell, A., Davies, A., Dowding, D., Khan, N., & Davies, A. (2022) · Journal of Community Genetics · doi:10.1007/s12687-022-00579-y · Cited by 16
Swaab, R. I., Lount, R. B., Chung, S., & Brett, J. M. (2021) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2021.08.001 · Cited by 18
Qiao, X., Hou, G., Kang, J., Wang, Q., & Yin, Y. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.833796 · Cited by 18
van Groesen, N. R. A., Briaire, J. J., & Frijns, J. H. M. (2023) · Ear and Hearing · doi:10.1097/aud.0000000000001365 · Cited by 18
Xia, Y., Zhou, J., Xun, X., Johnston, L. J., Wei, T., Gao, R., Zhang, Y., Reddy, B., Liu, C., Kim, G., et al. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00754-z · Cited by 18
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
