The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 12,301–12,400 of 23,881
New Technologies for Stroke Rehabilitation
Iosa, M., Hesse, S., Oliviero, A., & Paolucci, S. (2013) · Stroke Research and Treatment · doi:10.1155/2013/815814 · Cited by 14
Li, C., Loßnitzer, D., Katus, H. A., & Buss, S. J. (2011) · International journal of cardiac imaging · doi:10.1007/s10554-011-9924-0 · Cited by 15
A Comparison of Hip versus Ankle Exercises in Elders and the Influence on Balance and Gait
Gras, L. Z., Levangie, P. K., Goodwin-Segal, M., & Lawrence, D. A. (2004) · Journal of Geriatric Physical Therapy · doi:10.1519/00139143-200408000-00001 · Cited by 15
Guo, X., & Slesnick, N. (2012) · Journal of Marital and Family Therapy · doi:10.1111/j.1752-0606.2012.00301.x · Cited by 15
Repeatability of anorectal manometry in healthy volunteers and patients
Otto, S. D., Clewing, J. M., Gröne, J., et al. (2013) · Journal of Surgical Research · doi:10.1016/j.jss.2013.06.008 · Cited by 15
Uemura, K., Oya, T., & Uchiyama, Y. (2013) · Human Movement Science · doi:10.1016/j.humov.2013.07.007 · Cited by 15
Predictors of Helping Behavior Toward Coworkers With Disabilities
Miller, B. K., & Werner, S. (2007) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2007.00275.x · Cited by 15
Shankman, S. A., & Klein, D. N. (2002) · Journal of Affective Disorders · doi:10.1016/s0165-0327(01)00302-0 · Cited by 15
Autobiographical memory specificity among people with recovered memories of childhood sexual abuse
Raymaekers, L., Smeets, T., Peters, M. J. V., & Merckelbach, H. L. G. J. (2010) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2010.03.004 · Cited by 18
Strunk, D. R., Stewart, M., Hollon, S. D., DeRubeis, R. J., FAWCETT, J. A., Amsterdam, J. D., & Shelton, R. C. (2009) · Psychological Medicine · doi:10.1017/s0033291709991553 · Cited by 15
Variability of Clinical Functional MR Imaging Results: A Multicenter Study
Wurnig, M. C., Rath, J., Klinger, N., et al. (2013) · Radiology · doi:10.1148/radiol.13121357 · Cited by 15
Understanding the common inter-rater reliability measures
Lim, S., Palethorpe, N., & Rodger, S. (2012) · International Journal of Therapy and Rehabilitation · doi:10.12968/ijtr.2012.19.9.488 · Cited by 11
A Comparison of Statistical Models for Calculating Reliability of the Hoffmann Reflex
Christie, A. D., Kamen, G., Boucher, J. P., et al. (2010) · Measurement in Physical Education and Exercise Science · doi:10.1080/1091367x.2010.495549 · Cited by 15
Tyrakowski, M., Janusz, P., Mardjetko, S. M., et al. (2014) · European Spine Journal · doi:10.1007/s00586-014-3595-1 · Cited by 15
The 5 min running field test: test and retest reliability on trained men and women
Dabonneville, M., Berthon, P., Vaslin, P., et al. (2003) · European Journal of Applied Physiology · doi:10.1007/s00421-002-0617-1 · Cited by 15
Jiang, W.-W., Li, C., Li, A.-H., et al. (2014) · Ultrasonics · doi:10.1016/j.ultras.2014.09.009 · Cited by 15
Ratings of Therapist Dynamic Activities and Alliance Early and Late in Psychotherapy.
Ackerman, S. J., Hilsenroth, M. J., & Knowles, E. S. (2005) · Psychotherapy · doi:10.1037/0033-3204.42.2.225 · Cited by 15
Eason, T. B., Gavel, C. R., Hawley, K. A., et al. (2014) · Muscle & Nerve · doi:10.1002/mus.24561 · Cited by 15
Grasp force matching and brain iron content estimated in vivo in older women
Adamo, D. E., Daugherty, A. M., & Raz, N. (2013) · Brain Imaging and Behavior · doi:10.1007/s11682-013-9284-6 · Cited by 15
Trammell, T. R., Flint, K., & Ramsey, C. J. (2012) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.k.01470 · Cited by 15
Zhou, H. L., Muellerleile, P. A., Ingram, D., et al. (2011) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998610381399 · Cited by 15
Muniz, F. W. M. G., Sena, K., de Oliveira, C., Veríssimo, D., Carvalho, R., & Martins, R. (2014) · International Journal of Dental Hygiene · doi:10.1111/idh.12112 · Cited by 15
Mishima, K., Umeda, H., Nakano, A., et al. (2013) · Journal of Cranio-Maxillofacial Surgery · doi:10.1016/j.jcms.2013.05.035 · Cited by 15
van der Heijde, D. M. F. M., Burmester, G. R., Melo-Gomes, J., et al. (2008) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2008.094375 · Cited by 15
Naredo, E., Acebes, C., Brito, E., et al. (2013) · The Journal of Rheumatology · doi:10.3899/jrheum.121103 · Cited by 15
Cross-Validation of the JSORRAT-II in Iowa
Ralston, C. A., Epperson, D. L., & Edwards, S. R. (2014) · Sexual Abuse · doi:10.1177/1079063214548074 · Cited by 15
Effects of cueing and collaboration on the acquisition of complex legal skills
Hummel, H. G. K., PAAS, F. G. W. C., & Koper, R. (2006) · British Journal of Educational Psychology · doi:10.1348/000709905x85629 · Cited by 14
The Effect of Repeated Measurements Using an Upper Extremity Robot on Healthy Adults
Finley, M. A., Dipietro, L., Ohlhoff, J., Whitall, J., Krebs, H. I., & Bever, C. T. (2009) · Journal of Applied Biomechanics · doi:10.1123/jab.25.2.103 · Cited by 15
Ramírez‐Esparza, N., Harris, K., Hellermann, J., Richard, C., Kuhl, P. K., & Reder, S. (2011) · Language Learning · doi:10.1111/j.1467-9922.2011.00631.x · Cited by 22
Faye, A., Dumont, A., Ndiaye, P., & Fournier, P. F. (2014) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzu018 · Cited by 15
Geraedts, H. A. E., Zijlstra, W., van Keeken, H. G., Zhang, W., & Stevens, M. J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0137668 · Cited by 15
Grams, A. E., Knoflach, M., Rehwald, R., et al. (2015) · American Journal of Neuroradiology · doi:10.3174/ajnr.a4350 · Cited by 15
Objective measurements of excess skin in post bariatric patients – inter-rater reliability
Biörserud, C., Olsén, M. F., Elander, A., & Wiklund, M. (2015) · Journal of Plastic Surgery and Hand Surgery · doi:10.3109/2000656x.2015.1089876 · Cited by 15
Consensus-based training and assessment model for general surgery
Szász, P., Louridas, M., de Montbrun, S. L., Harris, K. A., & Grantcharov, T. (2016) · British journal of surgery · doi:10.1002/bjs.10103 · Cited by 17
Eguchi, K., Kuruvilla, S., Ishikawa, J., Ogedegbe, G. G., Gerin, W., Schwartz, J. E., & Pickering, T. G. (2011) · Blood Pressure Monitoring · doi:10.1097/mbp.0b013e328346d669 · Cited by 15
The relationship between the Nasality Severity Index 2.0 and perceptual judgments of hypernasality
Bettens, K., Bodt, M. S. D., Maryn, Y., et al. (2016) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2016.05.011 · Cited by 15
International society for prosthetics and orthotics.
Je, E. (1978) · PubMed · Cited by 15
Reliability and responsiveness of gait initiation profiles in those with chronic ankle instability
Hartley, E. M., Hoch, M. C., & McKeon, P. O. (2016) · Gait & Posture · doi:10.1016/j.gaitpost.2016.06.022 · Cited by 15
Transcultural validation of the Oxford Shoulder Score for the French-speaking population
Tuton, D., Barbe, C., Salmon, J. H., et al. (2016) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2016.05.010 · Cited by 15
Youth Baseball Pitching Stride Length: Normal Values and Correlation With Field Testing
Fry, K. E., Pipkin, A., Wittman, K., et al. (2016) · Sports Health A Multidisciplinary Approach · doi:10.1177/1941738116679815 · Cited by 15
Lairez, O., Delmas, C., Fournier, P., Cassol, E., Méjean, S., Pascal, P., Petermann, A., Dambrin, C., Minville, V., Carrié, D., et al. (2016) · Journal of Nuclear Cardiology · doi:10.1007/s12350-016-0670-5 · Cited by 15
Nazary‐Moghadam, S., Salavati, M., Esteki, A., Akhbari, B., Keyhani, S., & Zeinalzadeh, A. (2017) · Journal of Bodywork and Movement Therapies · doi:10.1016/j.jbmt.2017.02.004 · Cited by 15
Hakim, C. H., Mijailovic, A., Lessa, T. B., Coates, J. R., Shin, C., Rutkove, S. B., & Duan, D. (2017) · PLoS ONE · doi:10.1371/journal.pone.0173557 · Cited by 15
Jayanthi, M., Dimino, J., Gersten, R. M., Taylor, M. J., Haymond, K. S., Smolkowski, K., & Newman-Gonchar, R. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2017.1327625 · Cited by 18
A Comparison of Long- vs. Short-Term Recall of Substance Use and HIV Risk Behaviors
Janssen, T., Braciszewski, J. M., Vose-O’Neal, A., et al. (2017) · Journal of Studies on Alcohol and Drugs · doi:10.15288/jsad.2017.78.463 · Cited by 15
Mehta, B., Griffiths, N., Spence, K., & Laing, S. (2017) · Journal of Paediatrics and Child Health · doi:10.1111/jpc.13589 · Cited by 16
Claeson, A., Gouveia-Figueira, S. C., Häggström, J., Fowler, C. J., & Nording, M. L. (2017) · Prostaglandins Leukotrienes and Essential Fatty Acids · doi:10.1016/j.plefa.2017.06.004 · Cited by 15
Inter-arch digital model vs. manual cast measurements: Accuracy and reliability
Kiviahde, H., Bukovac, L., Jussila, P., Pesonen, P., Sipilä, K., Raustia, A. M., & Pirttiniemi, P. M. (2017) · CRANIO® · doi:10.1080/08869634.2017.1344811 · Cited by 15
Hocking, I., & Vernon, D. J. (2017) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2017.08.001 · Cited by 17
Cenni, F., Schless, S., Bar‐On, L., Molenaers, G., Campenhout, A. V., Aertbeliën, E., Bruyninckx, H., Hanssen, B., & Desloovere, K. (2017) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2017.09.018 · Cited by 16
Michaelov, E., Armstrong, J. J., Nguyen, M., et al. (2017) · Journal of Glaucoma · doi:10.1097/ijg.0000000000000820 · Cited by 15
The Impact of Premeeting Talk on Group Performance
Yoerger, M. A., Allen, J. A., & Crowe, J. D. (2017) · Small Group Research · doi:10.1177/1046496417744883 · Cited by 15
In vivo exploration of retinal nerve fiber layer morphology in Parkinson’s disease patients
Visser, F., Vermeer, K. A., Ghafaryasl, B., Vlaar, A. M. M., Apostolov, V., Hubar, J. V. H., Weinstein, H. C., de Boer, J. F., & Berendse, H. W. (2018) · Journal of Neural Transmission · doi:10.1007/s00702-018-1872-6 · Cited by 15
Lang, A., & Schulze, R. K. W. (2018) · Clinical Oral Investigations · doi:10.1007/s00784-018-2414-1 · Cited by 15
Effects of motivational interviewing fidelity on substance use treatment engagement in primary care
Osilla, K. C., Watkins, K. E., D’Amico, E. J., McCullough, C., & Ober, A. J. (2018) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2018.01.014 · Cited by 15
Cultural adaptation and validation of Patient and Observer Scar Assessment Scale for Turkish use
Seyyah, M., & Yurdalan, S. U. (2018) · Burns · doi:10.1016/j.burns.2018.02.026 · Cited by 15
Kulas, J., Klahr, R., & Knights, L. (2018) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000459 · Cited by 22
Hutin, É., Ghédira, M., Loche, C.-M., et al. (2018) · Topics in Stroke Rehabilitation · doi:10.1080/10749357.2018.1460932 · Cited by 15
Gibbon, K. C., Debuse, D., Hibbs, A. E., & Caplan, N. (2017) · Journal of Ultrasound in Medicine · doi:10.7863/ultra.16.03059 · Cited by 16
Akbari, A., Broeck, D. V., Benoy, I. H., Padalko, E. Y., Bogers, J., & Arbyn, M. (2018) · Virology Journal · doi:10.1186/s12985-018-1076-6 · Cited by 15
Amédro, P., Bredy, C., Guillaumont, S., et al. (2019) · International journal of cardiac imaging · doi:10.1007/s10554-018-01516-2 · Cited by 15
Bureau, N. J., Destrempes, F., Acid, S., Lungu, E., Moser, T., Michaud, J., & Cloutier, G. (2019) · Journal of Ultrasound in Medicine · doi:10.1002/jum.14964 · Cited by 15
Fung, Y. L., Ng, K. E. T., Vogrin, S. J., Meade, C. E., Ngo, M. M., Collins, S. J., & Bowden, S. C. (2019) · Journal of Alzheimer s Disease · doi:10.3233/jad-181172 · Cited by 17
Global Emergency Medicine: A Review of the Literature From 2015
Becker, T. K., Hansoti, B., Bartels, S. A., et al. (2016) · Academic Emergency Medicine · doi:10.1111/acem.12999 · Cited by 15
Barahona, J. D., Molina‐García, J., & Monfort-Pañego, M. (2019) · Retos · doi:10.47197/retos.v0i35.63355 · Cited by 15
Accuracy of School Staff‐Measured Height and Weight Used for Body Mass Index Screening and Reporting
Thompson, H. R., Linchey, J. K., King, B., Himes, J. H., & Madsen, K. A. (2019) · Journal of School Health · doi:10.1111/josh.12788 · Cited by 15
Hedelin, H., Larnert, P., Hebelka, H., et al. (2019) · Journal of Children s Orthopaedics · doi:10.1302/1863-2548.13.180195 · Cited by 15
Morphometric changes at the craniocervical junction during childhood
Bapuraj, J. R., Bruzek, A. K., Tarpeh, J., et al. (2019) · Journal of Neurosurgery Pediatrics · doi:10.3171/2019.4.peds1968 · Cited by 15
Ishimatsu, K., Nishie, A., Takayama, Y., et al. (2019) · European Journal of Radiology · doi:10.1016/j.ejrad.2019.108656 · Cited by 15
Safety perception of stairs with integrated lighting
de Perre, L. V., Danschutter, S., Janssens, K., et al. (2019) · Building and Environment · doi:10.1016/j.buildenv.2019.106389 · Cited by 15
Simons, J. D. (2019) · Professional School Counseling · doi:10.1177/2156759x19873902 · Cited by 16
Hassel, S., Sharma, G. B., Alders, G. L., Davis, A. D., Arnott, S. R., Frey, B. N., Hall, G. B., Harris, J. K., Lam, R. W. M., Milev, R. V., et al. (2019) · Human Brain Mapping · doi:10.1002/hbm.24883 · Cited by 14
Näf, O. B., Bauer, C. M., Zange, C., & Rast, F. M. (2019) · Gait & Posture · doi:10.1016/j.gaitpost.2019.12.011 · Cited by 15
Concurrent Imaging of Markers of Current Flow and Neurophysiological Changes During tDCS
Jog, M., Jann, K., Yan, L., Huang, Y., Parra, L. C., Narr, K. L., Bikson, M., & Wang, D. J. J. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00374 · Cited by 15
Nakajima, S., Sato, I., Soejima, T., Koh, K., Kato, M., Okamoto, Y., Imamura, T., Maeda, M., Ishida, Y., Manabe, A., et al. (2020) · BMC Pediatrics · doi:10.1186/s12887-020-02287-3 · Cited by 15
Wuennemann, F., Kintzelé, L., Braun, A., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-71311-4 · Cited by 15
Cardones, A. R., Hall, R. P., Sullivan, K. M., et al. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.07.031 · Cited by 15
Reliability and accuracy of heart rate variability metrics versus ECG segment duration
McNames, J., Thong, T., & Goldstein, B. (2004) · doi:10.1109/iembs.2003.1279574 · Cited by 15
Dynamic Probabilistic Graph Convolution for Facial Action Unit Intensity Estimation
Song, T., Cui, Z., Wang, Y., Zheng, W., & Ji, Q. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00481 · Cited by 16
Vita, M. J. D., Buckheit, K. A., Gilmour, C. E., et al. (2021) · Pain Medicine · doi:10.1093/pm/pnab290 · Cited by 15
Grimm, M. R., Lazariotou, M., Kircher, S., Höfelmayr, A., Germer, C. T., von Rahden, B. H. A., Waaga-Gasser, A. M., & Gasser, M. (2010) · Analytical Cellular Pathology · doi:10.1155/2010/891869 · Cited by 15
Oral-Hygiene-Related Mobile Apps in the French App Stores: Assessment of Functionality and Quality
Carrouel, F., Bourgeois, D. M., Clément, C., Tardivo, D., Martinon, P., Guiral, S., Lan, R., Viennot, S., Dussart, C., & Fraticelli, L. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19127293 · Cited by 19
Burns, S. A., Cleland, J. A., Carpenter, K., & Mintken, P. E. (2015) · Physical Therapy in Sport · doi:10.1016/j.ptsp.2015.07.001 · Cited by 15
Chow, T. W., Takeshita, S., Honjo, K., Pataky, C. E., Jacques, P. L. S., Kusano, M., Caldwell, C. B., Ramirez, J., Black, S. E., & Verhoeff, N. P. L. G. (2007) · BMC Nuclear Medicine · doi:10.1186/1471-2385-7-2 · Cited by 15
Keglovits, M., Somerville, E. K., & Stark, S. L. (2015) · American Journal of Occupational Therapy · doi:10.5014/ajot.2015.015248 · Cited by 15
Confiabilidade do Teste WPPSI-R na avaliação do desenvolvimento cognitivo de pré-escolares
Méio, M. D. B. B., Lopes, C. D. S., Sichieri, R., et al. (2001) · Cadernos de Saúde Pública · doi:10.1590/s0102-311x2001000100010 · Cited by 15
Chung, S., Domino, M. E., Jackson, E. W., & MORRISSEY, J. P. (2008) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-007-9105-z · Cited by 15
The influence of leg length difference on clinical outcome after revision TKA
Mufty, S., Vandenneucker, H., & Bellemans, J. (2012) · The Knee · doi:10.1016/j.knee.2012.09.007 · Cited by 15
Rasmussen, P. W., Wheeler, W. E., Moser, T. J., et al. (2001) · Journal of Wildlife Management · doi:10.2307/3803022 · Cited by 15
Pouchot, J., Trudeau, E., Hellot, S., et al. (2005) · Quality of Life Research · doi:10.1007/s11136-004-5682-y · Cited by 15
Dreher, A., Rader, T., Patscheider, M., Klemens, C., Schmidt, M., Baker, F. C., & de la Chaux, R. (2008) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-008-0750-5 · Cited by 15
Lindholm, T. L., Botes, L., Engman, E.-L., Frank, A., Jonsson, T., Svensson, L., & Julin, P. (2009) · BMC Medical Imaging · doi:10.1186/1471-2342-9-15 · Cited by 16
Validation of the Test Your Memory (F-TYM Test) in a French Memory Clinic Population
Postel‐Vinay, N., Hanon, O., Clerson, P., Brown, J. M., Ménard, J., Paillaud, É., Alonso, E., Pasquier, F., Pariel, S., Belliard, S., et al. (2014) · The Clinical Neuropsychologist · doi:10.1080/13854046.2014.934716 · Cited by 15
Barker, R. N., Amsters, D. I., Kendall, M. D., et al. (2007) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2007.02.032 · Cited by 15
Temporal Stability of the Wisconsin Card Sorting Test in a Chronic Traumatic Brain Injury Sample
Greve, K. W., Love, J. M., Sherwin, E. D., Mathias, C. W., Houston, R. J., & Brennan, A. M. (2002) · Assessment · doi:10.1177/1073191102009003006 · Cited by 15
Interrater reliability of scoring parameters for the Design Fluency Test.
Woodard, J. L., Axelrod, B. N., & Henry, R. R. (1992) · Neuropsychology · doi:10.1037//0894-4105.6.2.173 · Cited by 15
Heritability of Intracortical Inhibition and Facilitation
Pellicciari, M. C., Veniero, D., Marzano, C., Moroni, F., Pirulli, C., Curcio, G., Ferrara, M., Miniussi, C., Rossini, P. M., & Gennaro, L. D. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.2112-09.2009 · Cited by 18
Reproducibility and validity of the stair-climb test for fire fighters
Plat, M.-C. J., Frings‐Dresen, M. H. W., & Sluiter, J. K. (2010) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-010-0518-2 · Cited by 15
Bennini, J. R., Faro, C. B. D. A., Marussi, E. F., Barini, R., & Peralta, C. F. A. (2009) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.7517 · Cited by 15
Development and reliability of the Motivation for Change Questionnaire
Gard, G., Rivano, M., & Grahn, B. E. M. (2005) · Disability and Rehabilitation · doi:10.1080/09638280500052682 · Cited by 17
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
