The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 13,801–13,900 of 23,881
Exploring the Effects of Encouragement in Educational Games
Kao, D., & Harrell, D. F. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2892335 · Cited by 13
Lee, M., Kim, S., Choi, H., et al. (2017) · Isokinetics and Exercise Science · doi:10.3233/ies-160626 · Cited by 11
Das Basler Interview für Psychosen (BIP): Struktur, Reliabilität und Validität
Riecher‐Rössler, A., Ackermann, T., Uttinger, M., Ittig, S., Koranyi, S., Rapp, C., Bugra, H., & Studerus, E. (2015) · Fortschritte der Neurologie · Psychiatrie · doi:10.1055/s-0034-1398999 · Cited by 14
Reliability and clinical validity of a technique to assess lifetime illicit injection drug use.
Pieper, B. A., Templin, T., Birk, T. J., et al. (2008) · PubMed · Cited by 11
Smith, D. A., Troy, J. B., & Hirsch, P. L. (2020) · doi:10.18260/1-2--13247 · Cited by 11
Interhemispheric Inhibition Measurement Reliability in Stroke: A Pilot Study
Cassidy, J. M., Chu, H., Chen, M., Kimberley, T. J., & Carey, J. R. (2016) · Neuromodulation Technology at the Neural Interface · doi:10.1111/ner.12459 · Cited by 11
Almuhizi, Y., Leser, A., & Pegelow, M. (2016) · European Journal of Orthodontics · doi:10.1093/ejo/cjw046 · Cited by 11
A Study on Standardization of K-WPPSI-IV: Analyses of Reliability and Validity
Park, H., Lee, K.-O., Lee, S., et al. (2016) · Korean Journal of Childcare and Education · doi:10.14698/jkcce.2016.12.04.111 · Cited by 11
Cost-effectiveness analysis of single-session walk-in counselling
Lamsal, R., Stalker, C., Cait, C.-A., Riemer, M., & Horton, S. (2017) · Journal of Mental Health · doi:10.1080/09638237.2017.1340619 · Cited by 11
Intersession reliability of self-selected and narrow stance balance testing in older adults
Riemann, B. L., & Piersol, K. L. (2016) · Aging Clinical and Experimental Research · doi:10.1007/s40520-016-0687-2 · Cited by 11
Bekiroğlu, N., Bakkal, M., Özbay, G., Karadeniz, P. G., & Kargül, B. (2016) · Pediatric Dental Journal · doi:10.1016/j.pdj.2016.09.003 · Cited by 11
Rosenblum, U., & Melzer, I. (2016) · Neurology Report · doi:10.1097/npt.0000000000000161 · Cited by 11
Marchitelli, R., Collignon, O., & Jovicich, J. (2017) · Brain Connectivity · doi:10.1089/brain.2016.0450 · Cited by 11
Dynamic Behavior Analysis via Structured Rank Minimization
Georgakis, C., Panagakis, Y., & Pantić, M. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0985-3 · Cited by 11
Splett, J. W., Raborn, A., Lane, K. L., Binney, A. J., & Chafouleas, S. M. (2017) · Psychological Assessment · doi:10.1037/pas0000458 · Cited by 11
Reproducibility of scapular muscle activity in isokinetic shoulder flexion and extension
Wochatz, M., Rabe, S., Wolter, M., Engel, T., Mueller, S., & Mayer, F. (2017) · Journal of Electromyography and Kinesiology · doi:10.1016/j.jelekin.2017.04.006 · Cited by 11
Bunford, N., Kinney, K. L., Michael, J., & Klumpp, H. (2017) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2017.04.007 · Cited by 11
Teachers’ Literal and Inferential Talk in Early Childhood and Special Education Classrooms
Sembiante, S. F., Dynia, J., Kaderavek, J., & Justice, L. M. (2018) · Early Education and Development · doi:10.1080/10409289.2017.1362916 · Cited by 10
The Use of the Social Cognition and Object Relations Scale in a Primary Care Setting
Richardson, L. A., Porcerelli, J. H., Dauphin, V. B., Morris, P. A., & Murdoch, W. (2017) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1369984 · Cited by 11
Hong, C., Lin, C., Yang, J.-M., et al. (2017) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-017-2830-8 · Cited by 11
Moghadam, M., Salavati, M., Sahaf, R., Rassouli, M., Moghadam, M., & Kamrani, A. A. A. (2017) · Journal of Cross-Cultural Gerontology · doi:10.1007/s10823-017-9340-6 · Cited by 11
van Kernebeek, W. G., de Schipper, A. W., Savelsbergh, G. J. P., et al. (2017) · Measurement in Physical Education and Exercise Science · doi:10.1080/1091367x.2017.1399891 · Cited by 11
Reliability of the Apathy Evaluation Scale in Myotonic Dystrophy Type 1
Gallais, B., Gagnon, C., Côté, I., et al. (2017) · Journal of Neuromuscular Diseases · doi:10.3233/jnd-170274 · Cited by 11
Focused Evidence-Based Medicine Curriculum for Trainees in Neonatal-Perinatal Medicine
Pammi, M., Lingappan, K., Carbajal, M. M., et al. (2017) · MedEdPORTAL · doi:10.15766/mep_2374-8265.10664 · Cited by 11
Jasmine, F., Shinkle, J., Sabarinathan, M., Ahsan, H., Pierce, B. L., & Kibriya, M. G. (2018) · American Journal of Human Biology · doi:10.1002/ajhb.23118 · Cited by 11
Fouche, P. F., Stein, C. O. A., Simpson, P., Carlson, J. N., Zverinova, K. M., & Doi, S. A. R. (2018) · Prehospital Emergency Care · doi:10.1080/10903127.2017.1423139 · Cited by 11
The role of contrast-enhanced computed tomography to detect renal stones
Odenrick, A., Kartalis, N., Voulgarakis, N., Morsbach, F., & Loizou, L. (2018) · Abdominal Radiology · doi:10.1007/s00261-018-1778-7 · Cited by 12
Evaluation of gait symmetry using a tri-axial accelerometer in stroke patients
Terui, Y., Suto, E., Konno, Y., Kubota, K., Iwakura, M., Satou, M., Nitta, S., Hasegawa, K., Satake, M., & Shioya, T. (2018) · Neurorehabilitation · doi:10.3233/nre-172235 · Cited by 13
Assessment of epicardial fat and carotid intima media thickness in gestational hypertension
Altın, C., Yılmaz, M., Ozsoy, H. M., Gezmiş, E., Balcı, S., Tekindal, M. A., Sade, L. E., & Müderrisoğlu, H. (2018) · Journal of obstetrics and gynaecology research · doi:10.1111/jog.13631 · Cited by 11
Assessing the complexity of handwritten signatures
Stern, H. S., Angel, M., Cavanaugh, M., Zhu, S., & Lai, E. L. (2018) · Law Probability and Risk · doi:10.1093/lpr/mgy007 · Cited by 11
Nagano, R. C. D. R., Biasotto‐Gonzalez, D. A., da Costa, G. L., et al. (2018) · Neurourology and Urodynamics · doi:10.1002/nau.23595 · Cited by 11
Chen, L., Sun, P., Hao, Q., Yin, W., Xu, B., Ma, C., Stemmer, A., Fu, C., Wang, M., & Lu, J. (2018) · Oncotarget · doi:10.18632/oncotarget.25279 · Cited by 11
High reliability of morphokinetic annotations among embryologists
Martínez, M., Santaló, J., Rodríguez, A., et al. (2018) · Human Reproduction Open · doi:10.1093/hropen/hoy009 · Cited by 11
Oledzka, M., Kaplan, S. L., Sweeney, J. K., et al. (2018) · Pediatric Physical Therapy · doi:10.1097/pep.0000000000000510 · Cited by 11
Lewey, J. H., Kivisalu, T. M., & Giromini, L. (2018) · Journal of Personality Assessment · doi:10.1080/00223891.2018.1476361 · Cited by 11
Flavell, C. A., Marshman, L. A. G., & Gordon, S. J. (2018) · Ultrasound · doi:10.1177/1742271x18785403 · Cited by 11
How to Obtain Efficient High Reliabilities in Assessing Texts: Rubrics vs Comparative Judgement
Goossens, M., & Mæyer, S. D. (2018) · Communications in computer and information science · doi:10.1007/978-3-319-97807-9_2 · Cited by 11
Häske, D., Beckers, S. K., Hofmann, M., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0202795 · Cited by 11
Reliability of Electromyographic Assessment of Biceps Brachii and Triceps Brachii in Cricketers
Singla, D., Hussain, M. E., Bhati, P., Moiz, J. A., Ahmad, I., Verma, S., & Ali, K. (2018) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2018.04.001 · Cited by 11
Interrater Reliability of the Braden and Braden Q by Skin Champion Nurses
Riccioni, N., Berlanga, R., Hagan, J. L., et al. (2018) · Journal of Pediatric Nursing · doi:10.1016/j.pedn.2018.09.014 · Cited by 11
Berger, D. M. S., Wassenberg, R. M., Jóźwiak, K., et al. (2018) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2018.10.529 · Cited by 11
Overcoming metallic artefacts from orthopaedic wrist volar plating on a low-field MRI scanner
Stecco, A., Arioli, R., Buemi, F., et al. (2018) · La radiologia medica · doi:10.1007/s11547-018-0973-5 · Cited by 11
Bräuninger, T., Debus, M., Müller, J. M., et al. (2019) · Jahrbuch für Handlungs- und Entscheidungstheorie · doi:10.1007/978-3-658-23997-8_4 · Cited by 11
Maillard, P., Dimaggio, G., Berthoud, L., de Roten, Y., Despland, J., & Kramer, U. (2019) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12219 · Cited by 11
Reliability of different behavioural tests for growing pigs on-farm
Czycholl, I., Menke, S., Straßburg, C., & Krieter, J. (2019) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2019.02.004 · Cited by 11
Development of a new index of strength in adults with intellectual and developmental disabilities
Merchán-Baeza, J. A., Pérez‐Cruzado, D., González‐Sánchez, M., & Vargas, A. I. C. (2019) · Disability and Rehabilitation · doi:10.1080/09638288.2018.1543464 · Cited by 12
Tektas, O. Y., Kapsner, L. A., Lemmer, M., Bouna-Pyrrou, P., Lewczuk, P., Lenz, B., & Kornhuber, J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212167 · Cited by 11
Lagerberg, T. B., Hellström, A., Lundberg, E., & Hartelius, L. (2019) · Journal of Speech Language and Hearing Research · doi:10.1044/2018_jslhr-s-18-0018 · Cited by 12
Pizzoferrato, A., Timoh, K. N., Bader, G., Fort, J., Fritel, X., & Fauconnier, A. (2019) · International Urogynecology Journal · doi:10.1007/s00192-019-03933-w · Cited by 11
Harris, D., Wilson, M., Buckingham, G., & Vine, S. J. (2019) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2019.04.014 · Cited by 12
The Reliability and Validity of the Action Fluency Test in Healthy College Students
Ross, T. P., O’Connor, S., Holmes, G., Fuller, B., & Henrich, M. (2019) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acz016 · Cited by 11
Zhao, D.-L., Li, C., Chen, X., Ju, S., Deng, G., Xie, B., & Teng, G. (2019) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2019.04.020 · Cited by 11
Intra- and Inter-rater Agreement in a Subjective Speech Quality Assessment Task in Crowdsourcing
Jiménez, R. Z., Casanovas, A. L., Naderi, B., et al. (2019) · doi:10.1145/3308560.3317084 · Cited by 11
DiTA: a database of diagnostic test accuracy studies for physiotherapists
Kaizik, M. A., Hancock, M. J., & Herbert, R. (2019) · Journal of physiotherapy · doi:10.1016/j.jphys.2019.05.006 · Cited by 11
Buthmann, J. L., Finik, J. M., Ventura, G., Zhang, W., Shereen, A. D., & Nomura, Y. (2019) · Biological Psychology · doi:10.1016/j.biopsycho.2019.107716 · Cited by 10
Chalian, M., Schauwecker, N., Cai, A. R., Dessouky, R., Fey, N. P., Xi, Y., Chhabra, A. B., & Wells, J. E. (2019) · Skeletal Radiology · doi:10.1007/s00256-019-03263-4 · Cited by 11
Said, M., der Leeuw, R. J.-V., Spek, B., Brand, P. L. P., & van Dijk, N. (2019) · Perspectives on Medical Education · doi:10.1007/s40037-019-00527-6 · Cited by 11
Yağcı, G., Erel, S., & Okunakol, V. (2019) · Foot and Ankle Surgery · doi:10.1016/j.fas.2019.08.002 · Cited by 11
Wick, E. H., Johnson, K., DeMarre, K., Faherty, A., Parikh, S. R., & Horn, D. L. (2019) · Otolaryngology · doi:10.1177/0194599819856299 · Cited by 11
Cortellini, A., Porzio, G., Cofini, V., Necozione, S., Parisi, A., Peris, F., Ravoni, G., Spinelli, G., Masel, E. K., Berghoff, A. S., et al. (2019) · Palliative & Supportive Care · doi:10.1017/s1478951519000543 · Cited by 11
Roldán‐Jiménez, C., Martín‐Martín, J., & Cuesta‐Vargas, A. (2019) · JMIR mhealth and uhealth · doi:10.2196/13640 · Cited by 11
Giromini, L., Viglione, D. J., Vitolo, E., et al. (2019) · Scandinavian Journal of Psychology · doi:10.1111/sjop.12585 · Cited by 11
Payá, F. J. M., Díaz, J. R., & Payá, J. J. M. (2019) · Quantitative InfraRed Thermography Journal · doi:10.1080/17686733.2019.1700697 · Cited by 11
Grèze, C., Guttmann, A., Pons‐Rejraji, H., Vasson, M., Lornage, J., Ouchchane, L., & Brugnon, F. (2019) · Basic and Clinical Andrology · doi:10.1186/s12610-019-0098-2 · Cited by 11
Yan, Y., Gen, W., Wan, H., et al. (2020) · Medicine · doi:10.1097/md.0000000000018508 · Cited by 11
Dittrich, F., Beck, S., Harren, A. K., Reinecke, F., Serong, S., Jung, J., Back, D. A., Wolf, M. A., & Landgraeber, S. (2020) · JMIR mhealth and uhealth · doi:10.2196/17085 · Cited by 11
Quantitative sensory testing of periauricular skin in healthy adults
Lin, W., Zhou, F., Yu, L., Wan, L., Yuan, H., Wang, K., & Svensson, P. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-60724-w · Cited by 12
Winter peaks in web-based public inquiry into epistaxis
Liu, D. T., Besser, G., Parzefall, T., Riss, D., & Mueller, C. A. (2020) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-020-05915-x · Cited by 11
Fiabilidad y Validez de la Batería TGMD-2 en Población Española
Pérez, C. A., Carral, J. M. C., Sánchez-Lastra, M. A., Carballo-Roales, A. I., Domínguez-Meis, F., & Redondo, L. (2018) · Revista Iberoamericana de Diagnóstico y Evaluación - e Avaliação Psicológica · doi:10.21865/ridep50.1.02 · Cited by 11
Lisee, C. M., McGrath, M. L., Kuenze, C. M., Zhang, M., Salzler, M. J., Driban, J. B., & Harkey, M. S. (2020) · Journal of Sport Rehabilitation · doi:10.1123/jsr.2019-0476 · Cited by 11
Lin, S.-W., Liu, Y.-H., & Huang, E. Y. (2020) · Information Technology and People · doi:10.1108/itp-05-2019-0251 · Cited by 12
Development and Linguistic Cue Analysis of the State-Trait Cheerfulness Inventory–Short Form
Lau, C., Chiesi, F., Hofmann, J., Saklofske, D. H., & Ruch, W. (2020) · Journal of Personality Assessment · doi:10.1080/00223891.2020.1779733 · Cited by 11
Yıldız, A., Yıldız, R., Çelik, H. İ., et al. (2020) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2020.1805832 · Cited by 11
Schidlowski, M., Stirnberg, R., Stöcker, T., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-74680-y · Cited by 11
Automated abstraction of myocardial perfusion imaging reports using natural language processing
Zheng, C., Sun, B., Wu, Y., et al. (2020) · Journal of Nuclear Cardiology · doi:10.1007/s12350-020-02401-z · Cited by 11
Magnetic susceptibility imaging of human habenula at 3 T
Yoo, S., Kim, J.-W., Schenck, J. F., & Lee, S. (2020) · Scientific Reports · doi:10.1038/s41598-020-75733-y · Cited by 10
Dueñas, L., Arnal‐Gómez, A., Aparicio, I., Balasch-Bernat, M., López, L., González, J. C., Camallonga, C. S., & Alcántara, E. (2020) · Clinical Biomechanics · doi:10.1016/j.clinbiomech.2020.105252 · Cited by 11
Mao, Z., Duan, Y., Yao, Y., et al. (2021) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102454 · Cited by 11
Chen, H., Kumar, V. A., Johnson, J. M., Chen, M., Noll, K. R., Hou, P., Prabhu, S. S., Schomer, D. F., & Liu, H. (2021) · Magnetic Resonance in Medicine · doi:10.1002/mrm.28690 · Cited by 11
Ahrend, M., Rühle, M., Springer, F., et al. (2021) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-021-03831-1 · Cited by 11
Barwick, T. D., Orton, M. R., Koh, D., Kaiser, M., Rockall, A., Tunariu, N., Blackledge, M., & Messiou, C. (2021) · British Journal of Radiology · doi:10.1259/bjr.20200682 · Cited by 11
Kennes, A., Peeters, S. C. T., Janssens, M., Reijnders, J. S. A. M., Simons, M., Lataster, J., & Jacobs, N. (2021) · Psychologica Belgica · doi:10.5334/pb.799 · Cited by 19
Chen, X., Wang, D., Zhou, H. X., Zhu, R., Tian, Y., Du, Y. X., Chen, J. J., Chen, D. C., Wang, L., & Zhang, X. (2021) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2021.04.010 · Cited by 11
Velten, J., Hirschfeld, G. H. F., Meyers, M., & Margraf, J. (2021) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2021.03.001 · Cited by 10
Idiosyncratic Characteristics of Postural Sway in Normal and Perturbed Standing
Sakanaka, T. E., Lakie, M., & Reynolds, R. F. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.660470 · Cited by 11
Mokhtarinia, H. R., Shahbazi, A., Abma, F. I., et al. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.1931483 · Cited by 11
Suzuki, G., Ichibayashi, R., Yamamoto, S., Serizawa, H., Nakamichi, Y., Watanabe, M., & Honda, M. (2021) · Renal Failure · doi:10.1080/0886022x.2021.1943439 · Cited by 12
Radovanović, G., Kunz, J., Böhm, S., Arampatzis, A., & Legerlotz, K. (2021) · Journal of Sports Sciences · doi:10.1080/02640414.2021.1959981 · Cited by 11
Barry, N., Rowshanfarzad, P., Francis, R. J., et al. (2021) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-021-01049-4 · Cited by 11
Mazeres, F., Brinkmann, K., & Richter, M. M. (2021) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2021.08.004 · Cited by 11
Which comes first? Modeling the longitudinal association between mindfulness and neuroticism
Wang, H.-H., Wang, M., Wen, Z., Wang, Y., & Fang, J. (2021) · Personality and Individual Differences · doi:10.1016/j.paid.2021.111226 · Cited by 13
Marriott, B. R., Cho, E., Tugendrajch, S. K., Kliethermes, M. D., McMillen, J. C., Proctor, E. K., & Hawley, K. M. (2021) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-021-01169-9 · Cited by 16
Świerczyńska-Mróz, K., Krajewski, P. K., Reszke, R., et al. (2021) · Advances in Dermatology and Allergology · doi:10.5114/ada.2021.107271 · Cited by 11
Evolutionary Personality Psychology
Lewis, D. M. G., Al-Shawaf, L., & Buss, D. M. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108264822.022 · Cited by 11
COVID-19 and obsessive-compulsive symptoms in a large multi-site college sample
Berman, N. C., Fang, A., Hoeppner, S. S., Reese, H. E., Siev, J., Timpano, K. R., & Wheaton, M. G. (2022) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2022.100727 · Cited by 13
Kawahara, T., Sugita, Y., Momose, H., Szymanski, K. M., Hida, E., & Yamazaki, A. (2019) · Pediatrics International · doi:10.1111/ped.14003 · Cited by 11
Rahbar, Z. (2018) · Springer eBooks · doi:10.1007/978-3-319-56591-0_14-1 · Cited by 11
Link‐Sourani, D., Many, A., Ben‐Sira, L., Tarrasch, R., Bak, S., Kidron, D., Gordon, Z., Yagel, S., Harel, S., & Bashat, D. B. (2020) · Placenta · doi:10.1016/j.placenta.2020.05.001 · Cited by 12
Considerations when examining the psychometric properties of measurement instruments used in health
McKechnie, D., & Fisher, M. (2022) · Australian journal of advanced nursing · doi:10.37464/2020.392.481 · Cited by 12
Assessment of Patient-Reported Physical Function in Persons With Upper Extremity Amputation
Resnik, L. J., Stevens, P. M., Ni, P., et al. (2022) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000002044 · Cited by 11
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
