The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 9,801–9,900 of 23,881
Reisenzein, R., Hildebrandt, A., & Weber, H. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108264822.009 · Cited by 26
Boss, M. A., Snyder, B. S., Kim, E., et al. (2022) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.28093 · Cited by 23
Yan, M., Chen, L., Yang, M., Zhang, L., Niu, M., Wu, F., Chen, Y., Song, Z., Zhang, Y., Jiang, L., et al. (2022) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-022-01958-z · Cited by 22
Bai, Y., Pei, Y., Liu, W. V., Liu, W., Xie, S., Wang, X., Zhong, L., Chen, J., Zhang, L., Masokano, I. B., et al. (2022) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.28382 · Cited by 23
Inter-observer variance and agreement of wildlife information extracted from camera trap images
Zett, T., Stratford, K. J., & Weise, F. J. (2022) · Biodiversity and Conservation · doi:10.1007/s10531-022-02472-z · Cited by 23
Ji, C. W., Oh, H., Chang, K., et al. (2022) · Diversity · doi:10.3390/d14100797 · Cited by 23
Cross-cultural adaptation and validation of the Chinese version of the Sleep Health Index
Mu, Y., Wang, Y., Yuan, J., Knutson, K. L., Zhu, D., Izci‐Balserak, B., Wu, Y., & Zhu, B. (2022) · Sleep Health · doi:10.1016/j.sleh.2022.09.005 · Cited by 23
Oleksy, Ł., Królikowska, A., Mika, A., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192315885 · Cited by 23
Clinical Use of the CAPE-V Scales: Agreement, Reliability and Notes on Voice Quality
Nagle, K. F. (2022) · Journal of Voice · doi:10.1016/j.jvoice.2022.11.014 · Cited by 23
Hu, Y., Li, Q., Qiao, K., Zhang, X., Chen, B., & Yang, Z. (2022) · Human Brain Mapping · doi:10.1002/hbm.26194 · Cited by 23
A nomological network for misophonia in two German samples using the S-Five model for misophonia
Remmert, N., Jebens, A., Gruzman, R., Gregory, J. E., & Vitoratou, S. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.902807 · Cited by 32
Schwarz, A., Husain, A. A., Einaudi, L., Thürlimann, E., Läderach, J., Easthope, C. A., Held, J. P. O., & Luft, A. R. (2023) · Sensors · doi:10.3390/s23020624 · Cited by 25
A Digital Box and Block Test for Hand Dexterity Measurement: Instrument Validation Study
Prochaska, E., & Ammenwerth, E. (2023) · JMIR Rehabilitation and Assistive Technologies · doi:10.2196/50474 · Cited by 23
Vasak, C., Strbac, G. D., Huber, C. D., et al. (2013) · Clinical Implant Dentistry and Related Research · doi:10.1111/cid.12085 · Cited by 23
An observational study of teamwork skills in shift handover
Symons, N. R. A., Wong, H. W. L., Manser, T., Sevdalis, N., Vincent, C. A., & Moorthy, K. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2012.05.010 · Cited by 24
Warner, C. M., Brice, C. S., Esseling, P. G., Stewart, C. E., Mufson, L. H., & Herzig, K. A. (2013) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0498-0 · Cited by 24
Help that is not recognized: Harmful neglect of decision support systems
Althuizen, N., Reichel, A., & Wierenga, B. (2012) · Decision Support Systems · doi:10.1016/j.dss.2012.08.016 · Cited by 23
Goodman, G., Edwards, K., & Chung, H. (2012) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12001 · Cited by 24
Módis, L., Szalai, E., Németh, G., & Berta, A. (2010) · Cornea · doi:10.1097/ico.0b013e318200096a · Cited by 22
Back pain arising from schoolbag usage among primary schoolchildren
Adeyemi, A. J., Rohani, J. M., & Rani, M. R. A. (2014) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2014.06.001 · Cited by 23
Palmer, L. A., Johnston, S. S., Rousculp, M. D., Chu, B. C., Nichol, K. L., & Mahadevia, P. J. (2012) · Value in Health · doi:10.1016/j.jval.2011.12.015 · Cited by 24
Aati, O., Taylor, W. J., Siegert, R. J., Horne, A. M., House, M. E., Tan, P., Drake, J. M., Stamp, L. K., & Dalbeth, N. (2014) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2014-205671 · Cited by 24
Kongstad, T., Buchvald, F. F., Green, K., Lindblad, A., Robinson, T. E., & Nielsen, K. G. (2013) · Journal of Cystic Fibrosis · doi:10.1016/j.jcf.2013.05.012 · Cited by 23
Assessing certainty of activation or inactivation in test–retest fMRI studies
Maitra, R. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.073 · Cited by 23
Testing of the Patients’ Insights and Views of Teamwork (PIVOT) Survey: A validity study
Henry, B. W., Rooney, D. M., Eller, S., Vozenilek, J. A., & McCarthy, D. M. (2014) · Patient Education and Counseling · doi:10.1016/j.pec.2014.06.002 · Cited by 23
Lozano, D. C., & Twa, M. D. (2012) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-9940 · Cited by 23
Measuring standing balance in adults
Tesio, L., Rota, V., Longo, S., et al. (2013) · International Journal of Rehabilitation Research · doi:10.1097/mrr.0000000000000037 · Cited by 23
Kavvadias, T., Pelikan, S., Roth, P., Baeßler, K., & Schuessler, B. (2012) · International Urogynecology Journal · doi:10.1007/s00192-012-1837-5 · Cited by 23
Cowan, S., Hodges, P. W., & Bennell, K. L. (2001) · Physical Therapy in Sport · doi:10.1054/ptsp.2001.0056 · Cited by 23
Three-Dimensional Morphometric Study of the Trapezium Shape of the Trochlea Tali
Daud, R., Kadir, M. R. A., Izman, S., et al. (2013) · The Journal of Foot & Ankle Surgery · doi:10.1053/j.jfas.2013.03.007 · Cited by 23
Naturalness and interestingness of test images for visual quality evaluation
Halonen, R., Westman, S., & Oittinen, P. T. (2010) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.872390 · Cited by 23
Reliability of quantification measures of actinic keratosis
Chen, S. C., Hill, N. D., Veledar, E., et al. (2013) · British Journal of Dermatology · doi:10.1111/bjd.12591 · Cited by 23
Data-driven shrinkage of the spectral density matrix of a high-dimensional time series
Fiecas, M. J. A., & von Sachs, R. (2014) · Electronic Journal of Statistics · doi:10.1214/14-ejs977 · Cited by 23
Radiographic measurement reliability of lumbar lordosis in ankylosing spondylitis
Lee, J. S., Goh, T. S., Park, S. H., et al. (2012) · European Spine Journal · doi:10.1007/s00586-012-2575-6 · Cited by 23
Weck, F., Gropalis, M., Neng, J. M. B., & Witthöft, M. (2012) · International Journal of Behavioral Medicine · doi:10.1007/s12529-012-9276-8 · Cited by 23
Petersen, C. M., Johnson, R. D., Schuit, D., et al. (1994) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.1994.20.4.207 · Cited by 23
Boe, S. G., Antonowicz, N. M., Leung, V. W., et al. (2010) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2010.07.024 · Cited by 23
The Relation Between Co-rumination, Maternal Depressive Symptoms and Child Psychopathology
Grimbos, T., Granic, I., & Pepler, D. J. (2013) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-013-9342-9 · Cited by 26
Salivary Secretion and Occlusal Force in Patients with Unilateral Cerebral Stroke
Kawasaka, T., Shimodozono, M., 緒方, 敦., Tanaka, N., & Kawahira, K. (2010) · International Journal of Neuroscience · doi:10.3109/00207450802338754 · Cited by 24
Rábago, C. A., Lancaster, J. L., Narayana, S., Zhang, W., & Fox, P. T. (2009) · Clinical Neurophysiology · doi:10.1016/j.clinph.2009.04.020 · Cited by 23
Aesthetic evaluation after non‐invasive treatment for superficial basal cell carcinoma
Mosterd, K., Arits, A. H. M. M., Nelemans, P. J., & Kelleners-Smeets, N. W. J. (2011) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/j.1468-3083.2011.04347.x · Cited by 23
Klomp, A., Caan, M. W. A., Denys, D. A. J. P., Nederveen, A. J., & Reneman, L. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.07.038 · Cited by 23
Jenkins, N. D. M., Housh, T. J., Palmer, T. B., Cochrane, K. C., Bergstrom, H. C., Johnson, G. O., Schmidt, R. J., & Cramer, J. T. (2014) · Muscle & Nerve · doi:10.1002/mus.24505 · Cited by 26
Psychometric properties of a Dutch version of the Behavior Problems Inventory-01 (BPI-01)
Dumont, E. L. M., Kroes, D., Korzilius, H. P. L. M., et al. (2014) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2014.01.003 · Cited by 23
Gerdsri, P., & Kocaoglu, D. F. (2009) · Portland International Conference on Management of Engineering and Technology (PICMET) · doi:10.1109/picmet.2009.5262211 · Cited by 23
Hillstrom, H. J., Garg, R., Kraszewski, A. P., et al. (2014) · Journal of Applied Biomechanics · doi:10.1123/jab.2011-0094 · Cited by 23
Li, X., & Wang, H. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00383 · Cited by 23
Smeerdijk, M., Keet, R., de Haan, L., Barrowclough, C., LINSZEN, D. H., & Schippers, G. M. (2013) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2013.09.006 · Cited by 23
Work ability assessment in prolonged depressive illness
Slebus, F. G., Kuijer, P., Willems, J. H. B. M., Frings‐Dresen, M. H. W., & Sluiter, J. K. (2010) · Occupational Medicine · doi:10.1093/occmed/kqq079 · Cited by 23
Compton, M. T., Kelley, M. E., Pope, A., Smith, K., Broussard, B., Reed, T., DiPolito, J. A., Druss, B. G., Li, C. H., & Haynes, N. L. (2015) · Psychiatric Services · doi:10.1176/appi.ps.201300482 · Cited by 23
Emerging narrative discourse skills 18 months after traumatic brain injury in early childhood
Walz, N. C., Yeates, K. O., Taylor, H. G., Stancin, T., & Wade, S. L. (2011) · Journal of Neuropsychology · doi:10.1111/j.1748-6653.2011.02020.x · Cited by 28
Ejaculate quality in spotted hyenas: intraspecific variation in relation to life-history traits
Curren, L. J., Weldele, M. L., & Holekamp, K. E. (2013) · Journal of Mammalogy · doi:10.1644/12-mamm-a-057.1 · Cited by 23
Testing the Reliability of Neighborhood-Specific Measures of Physical Activity Among Canadian Adults
McCormack, G. R., Shiell, A., Doyle–Baker, P. K., Friedenreich, C. M., Sandalack, B. A., & Giles‐Corti, B. (2009) · Journal of Physical Activity and Health · doi:10.1123/jpah.6.3.367 · Cited by 23
Parenting Young Children with and Without Fragile X Syndrome
Sterling, A. M., Barnum, L., Skinner, D., Warren, S. F., & Fleming, K. (2012) · American Journal on Intellectual and Developmental Disabilities · doi:10.1352/1944-7558-117.3.194 · Cited by 23
Steinsvåg, K., Bråtveit, M., Moen, B. E., et al. (2007) · Occupational and Environmental Medicine · doi:10.1136/oem.2006.030528 · Cited by 23
Developmental themes in women's emotional experiences of motherhood
Luthar, S. S., Doyle, K., Suchman, N. E., & Mayes, L. C. (2001) · Development and Psychopathology · doi:10.1017/s0954579401001110 · Cited by 24
Bankstahl, U. S., & Görtelmeyer, R. (2013) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-11-86 · Cited by 23
John, M. T., Reissmann, D. R., Schierz, O., & Allen, P. (2008) · Acta Odontologica Scandinavica · doi:10.1080/00016350802077835 · Cited by 23
Muscle tone in different joint positions and at submaximal isometric torque levels
Alamäki, A., HÄKKINEN, A. H., Mälkiä, E., et al. (2007) · Physiological Measurement · doi:10.1088/0967-3334/28/8/003 · Cited by 23
Rating Scale, Standard Gamble, and Time Trade-off for People With Traumatic Spinal Cord Injuries
Lin, M., Hwang, H., Chung, K., et al. (2006) · Physical Therapy · doi:10.1093/ptj/86.3.337 · Cited by 23
Brief Rating of Aggression by Children and Adolescents (BRACHA): a reliability study.
Barzman, D. H., Mossman, D., Sonnier, L., et al. (2012) · PubMed · Cited by 23
Lim, J.-Y., Kim, T.-H., & Lee, J. (2015) · Journal of Physical Therapy Science · doi:10.1589/jpts.27.3119 · Cited by 23
Williams, D., Protopapa, E., Stöhr, K., et al. (2016) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-016-0867-4 · Cited by 23
Why the Propensity for Propensity Scores?*
Day, A. G. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001175 · Cited by 21
Calderwood, A. H., Logan, J. R., Zurfluh, M., et al. (2014) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000000028 · Cited by 23
Cao, X., Song, C., Guo, L., Jun-xia, Y., Deng, S., Xu, Y., Chen, X., Sapa, W. B., & Wang, K. (2015) · Medicine · doi:10.1097/md.0000000000001475 · Cited by 23
Perceived threat during deployment: Risk factors and relation to axis I disorders.
Mott, J. M., Graham, D. P., & Teng, E. J. (2011) · Psychological Trauma Theory Research Practice and Policy · doi:10.1037/a0025778 · Cited by 23
Single- vs. Double-Bundle Anterior Cruciate Ligament Reconstruction
Czamara, A., Królikowska, A., Szuba, Ł., et al. (2014) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000000638 · Cited by 23
Health Education Specialist Practice Analysis 2015 (HESPA 2015)
McKenzie, J. F., Dennis, D., Auld, M. E., et al. (2016) · Health Education & Behavior · doi:10.1177/1090198116639258 · Cited by 23
The test-retest reliability of four functional mobility tests in apparently healthy adults
Northgraves, M. J., Hayes, S. C., Marshall, P., Madden, L. A., & Vince, R. V. (2016) · Isokinetics and Exercise Science · doi:10.3233/ies-160614 · Cited by 24
Whitehurst, D. G. T., Mittmann, N., Noonan, V. K., Dvorak, M. F., & Bryan, S. (2016) · Quality of Life Research · doi:10.1007/s11136-016-1297-3 · Cited by 23
Exploring Gender Differences in the Utility of Strength-Based Risk Assessment Measures
Viljoen, S., Nicholls, T. L., Roesch, R. G., et al. (2016) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2016.1170739 · Cited by 23
Guo, J., Lau, A. H. Y., Chau, W., et al. (2016) · European Spine Journal · doi:10.1007/s00586-016-4590-5 · Cited by 23
Baque, E., Barber, L. A., Sakzewski, L., & Boyd, R. N. (2016) · Brain Injury · doi:10.3109/02699052.2016.1165869 · Cited by 21
Determinants of perceived success in management consulting
Bronnenmayer, M., Wirtz, B. W., & Göttel, V. (2016) · Management Research Review · doi:10.1108/mrr-06-2014-0145 · Cited by 32
Ribeiro, E., Cunha, C. A. C., Teixeira, A. S., Stiles, W. B., Pires, N., Santos, B., Basto, I., & Salgado, J. (2016) · Psychotherapy Research · doi:10.1080/10503307.2016.1208853 · Cited by 23
Borderline personality pathology and physical health: The role of employment.
Cruitt, P. J., Boudreaux, M. J., Jackson, J. J., & Oltmanns, T. F. (2016) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000211 · Cited by 23
Assessing adherence to multiple medications and in daily life among patients with multimorbidity
Inauen, J., Bierbauer, W., Lüscher, J., König, C., Tobias, R. M., Ihle, A., Zimmerli, L., Holzer, B. M., Battegay, E. J., Siebenhuener, K., et al. (2017) · Psychology and Health · doi:10.1080/08870446.2016.1275632 · Cited by 23
Effects of oral 3% hydrogen peroxide used as an emetic on the gastroduodenal mucosa of healthy dogs
Niedzwecki, A. H., Book, B. P., Lewis, K. M., et al. (2016) · Journal of Veterinary Emergency and Critical Care · doi:10.1111/vec.12558 · Cited by 23
Using electronic monitoring to record catches of sole (Solea solea) in a bottom trawl fishery
van Helmond, A. T. M., Chen, C., & Poos, J. J. (2016) · ICES Journal of Marine Science · doi:10.1093/icesjms/fsw241 · Cited by 23
Echegaray, S., Nair, V. S., Kadoch, M. A., Leung, A. N. C., Rubin, D. L., Gevaert, O., & Napel, S. A. (2016) · Tomography · doi:10.18383/j.tom.2016.00163 · Cited by 23
Reliability and Validity of START and LSI-R Assessments in Mental Health Jail Diversion Clients
Lowder, E. M., Desmarais, S. L., Rade, C. B., Johnson, K. L., & Dorn, R. A. V. (2017) · Assessment · doi:10.1177/1073191117704505 · Cited by 25
Alliance or Acquisition? A Mechanisms‐Based, Policy‐Capturing Analysis
Mellewigt, T., Thomas, A., Weller, I., & Zajac, E. J. (2017) · Strategic Management Journal · doi:10.1002/smj.2664 · Cited by 25
Joint moment-angle properties of the hip abductors and hip extensors
Kindel, C., & Challis, J. H. (2017) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2017.1323357 · Cited by 23
Pastorino, G. Q., Viau, A., Curone, G., Pearce‐Kelly, P., Faustini, M., Vigo, D., Mazzola, S., & Preziosi, R. F. (2017) · Veterinary Medicine International · doi:10.1155/2017/6585380 · Cited by 22
Song, S. H., Park, S., Chae, S. Y., et al. (2017) · Urology · doi:10.1016/j.urology.2017.05.044 · Cited by 23
Cone, J. A., Martin, T. M., Marcellin‐Little, D. J., et al. (2017) · American Journal of Veterinary Research · doi:10.2460/ajvr.78.8.900 · Cited by 23
Wright, H. H., O’Brien, V. H., Valdes, K. A., Koczan, B. J., MacDermid, J. C., Moore, E. S., & Finley, M. A. (2017) · Journal of Hand Therapy · doi:10.1016/j.jht.2017.04.003 · Cited by 23
Quantitative analysis of retinal perfusion in mice using optical coherence tomography angiography
Alnawaiseh, M., Brand, C., Bormann, E., Wistuba, J., Eter, N., & Heiduschka, P. (2017) · Experimental Eye Research · doi:10.1016/j.exer.2017.09.003 · Cited by 23
Validating bowel preparation scales
Heron, V., Parmar, R., Ménard, C., et al. (2017) · Endoscopy International Open · doi:10.1055/s-0043-119749 · Cited by 23
Phillips, B. M., Zhao, Y., & Weekley, M. J. (2017) · Early Education and Development · doi:10.1080/10409289.2017.1408371 · Cited by 28
Lee, P., Lu, W.-S., Liu, C.-H., Lin, H., & Hsieh, C. (2017) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acx123 · Cited by 23
Koopman, A. A., Blokpoel, R. G. T., van Eykern, L. A., et al. (2018) · Annals of Intensive Care · doi:10.1186/s13613-018-0359-9 · Cited by 23
Kutzner, K. P., Freitag, T. L., Bieger, R., et al. (2018) · Clinical Biomechanics · doi:10.1016/j.clinbiomech.2018.01.004 · Cited by 23
Froud, R., Fawkes, C. A., Foss, J. G. K., Underwood, M. R., & Carnes, D. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9828 · Cited by 25
Nguyen, X.-M., Quaden, R., Song, R. J., Ho, Y.-L. A., Honerlaw, J. P., Whitbourne, S. B., DuVall, S. L., Deen, J. E., Pyarajan, S., Moser, J., et al. (2018) · Journal of Health Research and Reviews · doi:10.4103/jhrr.jhrr_10_18 · Cited by 23
Measuring Tibial Component Rotation Following Total Knee Arthroplasty: What Is the Best Method?
Saffi, M., Spangehl, M. J., Clarke, H. D., & Young, S. W. (2018) · The Journal of Arthroplasty · doi:10.1016/j.arth.2018.10.022 · Cited by 23
Sledge runner fasciculus: anatomic architecture and tractographic morphology
Koutsarnakis, C., Kalyvas, A., Skandalakis, G. P., Karavasilis, E., Christidi, F., Komaitis, S., Velonakis, G., Liakos, F., Emelifeonwu, J. A., Giavri, Z., et al. (2019) · Brain Structure and Function · doi:10.1007/s00429-018-01822-4 · Cited by 23
Mitchell, S. L., Obremskey, W. T., Luly, J. R., et al. (2019) · Journal of Orthopaedic Trauma · doi:10.1097/bot.0000000000001445 · Cited by 23
Prevalence of thoracic aortic aneurysms and dilatations in patients with intracranial aneurysms
Laukka, D., Pan, E., Fordell, T., Alpay, K., Rahi, M., Hirvonen, J., Rinne, J., & Gunn, J. M. (2019) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2019.01.066 · Cited by 24
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review