The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 5,501–5,600 of 23,883
Stacchi, C., Bernardello, F., Spinato, S., Mura, R., Perelli, M., Lombardi, T., Troiano, G., & Canullo, L. (2022) · Clinical Oral Implants Research · doi:10.1111/clr.13959 · Cited by 50
Koutsoumpis, A., Ghassemi, S., Oostrom, J. K., Holtrop, D., van Breda, W., Zhang, T., & de Vries, R. E. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2023.108128 · Cited by 65
Validity and Reliability of Hallux Valgus Angle Measured on Digital Photographs
Nix, S. E., Russell, T. G., Vicenzino, B., & Smith, M. D. (2012) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2012.3841 · Cited by 51
Liu, X., Wang, X., Lin, S., Song, Q., Lao, X. Q., & Yu, I. T.-S. (2015) · PLoS ONE · doi:10.1371/journal.pone.0134627 · Cited by 55
Reliability and responsiveness of elbow trajectory tracking
Patten, C., Kothari, D. H., Whitney, J. M., Lexell, J. E., & Lum, P. S. (2003) · The Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2003.11.0487 · Cited by 51
Do your employees (collectively) trust you? The importance of trust climate beyond individual trust
Jiang, L., & Probst, T. M. (2015) · Scandinavian Journal of Management · doi:10.1016/j.scaman.2015.09.003 · Cited by 51
Greene, B. R., Rutledge, S. M., McGurgan, I. J., McGuigan, C., O’Connell, K., Caulfield, B. M., & Tubridy, N. J. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2435057 · Cited by 51
Barisch‐Fritz, B., Schmeltzpfenning, T., Plank, C., & Grau, S. (2014) · Ergonomics · doi:10.1080/00140139.2014.899629 · Cited by 51
Cadenhead, K. S. (2011) · Psychiatry Research · doi:10.1016/j.psychres.2011.04.011 · Cited by 51
Petersen, C., Johnson, R. D., & Schuit, D. (2000) · Manual Therapy · doi:10.1054/math.2000.0232 · Cited by 51
Schaaf, H., Wilbrand, J.-F., Boedeker, R., & Howaldt, H. (2010) · The Cleft Palate-Craniofacial Journal · doi:10.1597/09-026 · Cited by 51
Multicenter Pivotal Study of Vacuum-Assisted Precise Tissue Release for the Treatment of Cellulite
Kaminer, M. S., Coleman, W. P., Weiss, R. A., et al. (2015) · Dermatologic Surgery · doi:10.1097/dss.0000000000000280 · Cited by 51
A magnetic resonance imaging study of the entorhinal cortex in treatment-resistant depression
Furtado, C. P., Maller, J. J., & Fitzgerald, P. B. (2008) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2007.11.005 · Cited by 50
Borich, M. R., Wadden, K. P., & Boyd, L. A. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.009 · Cited by 51
Thien, L. M., & Razak, N. B. A. (2012) · Social Indicators Research · doi:10.1007/s11205-012-0077-x · Cited by 51
Therapeutic alliance, cohesion and outcome in a long-term analytic group. A preliminary study
Lorentzen, S., Sexton, H. C., & Høglend, P. A. (2004) · Nordic Journal of Psychiatry · doi:10.1080/08039480310000770 · Cited by 51
Test–retest stability of calibrated BOLD-fMRI in HIV− and HIV+ subjects
Ances, B. M., Vaida, F., Ellis, R. J., & Buxton, R. B. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.081 · Cited by 51
Taylor, J., Yon, B. A., & Johnson, R. K. (2014) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2014.02.029 · Cited by 51
Personality subtypes in adolescents with anorexia nervosa
Gazzillo, F., Lingiardi, V., Peloso, A. M., Giordani, S., Vesco, S., Zanna, V., Filippucci, L., & Vicari, S. (2013) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2013.03.006 · Cited by 54
Gay, A., Harbst, K., Hansen, D. K., et al. (2011) · The Journal Of Hand Surgery · doi:10.1016/j.jhsa.2011.07.027 · Cited by 51
Manual palpation of lumbo-pelvic landmarks: A validity study
Kilby, J., Heneghan, N. R., & Maybury, M. (2011) · Manual Therapy · doi:10.1016/j.math.2011.08.008 · Cited by 51
Nakamura, B. J., Selbo‐Bruns, A., Okamura, K. H., Chang, J. P., Slavin, L. A., & Shimabukuro, S. K. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.12.001 · Cited by 52
Decision-Making in Multiple Sclerosis Consultations in Italy: Third Observer and Patient Assessments
Pietrolongo, E., Giordano, A., Kleinefeld, M., Confalonieri, P. A., Lugaresi, A., Tortorella, C., Pugliatti, M., Radice, D., Goss, C., Heesen, C., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0060721 · Cited by 51
Validation of the Korean version of the Scoliosis Research Society-22 questionnaire
Lee, J. S., Lee, D. H., Suh, K. T., Kim, J. I., Lim, J. M., & Goh, T. S. (2011) · European Spine Journal · doi:10.1007/s00586-011-1872-9 · Cited by 52
Acute increases in synaptic GABA detectable in the living human brain: A [11C]Ro15-4513 PET study
Stokes, P. R. A., Myers, J. F., Kalk, N. J., Watson, B. J., Erritzøe, D., Wilson, S. J., Cunningham, V. J., Barros, D. A. R., Hammers, A., Turkheimer, F., et al. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.05.035 · Cited by 52
Relationship Between Corneal Hysteresis and Corneal Resistance Factor with Other Ocular Parameters
Rosa, N., Lanza, M., Bernardo, M. D., et al. (2014) · Seminars in Ophthalmology · doi:10.3109/08820538.2013.874479 · Cited by 51
HIV Prevalence Rates among Injection Drug Users in 96 Large US Metropolitan Areas, 1992–2002
Tempalski, B., Lieb, S., Cleland, C. M., Cooper, H. L. F., Brady, J. E., & Friedman, S. R. (2008) · Journal of Urban Health · doi:10.1007/s11524-008-9328-1 · Cited by 51
Global and Specific Behavioral Measures of Pain in Children With Cerebral Palsy
Hadden, K. L., & von Baeyer, C. L. (2005) · Clinical Journal of Pain · doi:10.1097/00002508-200503000-00005 · Cited by 51
Huth, K. C., Lussi, A., Gygax, M., Thum, M., Crispin, A., Paschos, E., Hickel, R., & Neuhaus, K. W. (2010) · Journal of Dentistry · doi:10.1016/j.jdent.2010.09.001 · Cited by 51
Reliable assessment of liver function using LiMAx
Jara, M., Bednarsch, J., Valle, E., et al. (2014) · Journal of Surgical Research · doi:10.1016/j.jss.2014.07.041 · Cited by 51
Schülein, S., Mendoza, S., Malzkorn, R., Harms, J., & Skwara, A. (2012) · Journal of Spinal Disorders & Techniques · doi:10.1097/bsd.0b013e318281608c · Cited by 50
Webb, C. A., DeRubeis, R. J., Dimidjian, S., Hollon, S. D., Amsterdam, J. D., & Shelton, R. C. (2012) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0027663 · Cited by 51
Tasca, G. A., Ritchie, K., Demidenko, N., Balfour, L., Krysanski, V. L., Weekes, K., Barber, A. M., Keating, L., & Bissada, H. (2012) · Psychotherapy Research · doi:10.1080/10503307.2012.717309 · Cited by 51
Public Responsiveness of Government Organizations: Testing a Preliminary Model
Yang, K., & Pandey, S. K. (2007) · Public Performance & Management Review · doi:10.2753/pmr1530-9576310204 · Cited by 52
Watson, J. C., McMullen, E., Prosser, M. C., & Bedard, D. L. (2010) · Psychotherapy Research · doi:10.1080/10503307.2010.518637 · Cited by 65
Pavão, S. L., dos Santos, A. N., Oliveira, A. B., & Rocha, N. A. C. F. (2013) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2013.11.028 · Cited by 51
Development and evaluation of a risk communication curriculum for medical students
Han, P. K. J., Joekes, K., Elwyn, G. J., Mazor, K. M., Thomson, R., Sedgwick, P. M., Ibison, J. M., & Wong, J. B. (2013) · Patient Education and Counseling · doi:10.1016/j.pec.2013.09.009 · Cited by 51
Tamburella, F., Scivoletto, G., Iosa, M., & Molinari, M. (2014) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-11-86 · Cited by 54
Measuring voluntary quadriceps activation: Effect of visual feedback and stimulus delivery
Luc, B. A., Harkey, M. S., Arguelles, G. D., et al. (2015) · Journal of Electromyography and Kinesiology · doi:10.1016/j.jelekin.2015.10.006 · Cited by 51
The Latarjet coracoid process transfer procedure: alterations in the neurovascular structures
Freehill, M. T., Srikumaran, U., Archer, K. R., et al. (2012) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2012.06.003 · Cited by 51
Shoulder muscle endurance: the development of a standardized and reliable protocol
Roy, J., Ma, B., MacDermid, J. C., & Woodhouse, L. J. (2011) · Sports Medicine Arthroscopy Rehabilitation Therapy & Technology · doi:10.1186/1758-2555-3-1 · Cited by 50
Weber, U., Zhao, Z., Rufibach, K., et al. (2014) · Arthritis & Rheumatology · doi:10.1002/art.39001 · Cited by 51
Unraveling the Relationship Between Family-Supportive Supervisor and Employee Performance
Wang, P., Walumbwa, F. O., Wang, H., & Aryee, S. (2013) · Group & Organization Management · doi:10.1177/1059601112472726 · Cited by 51
Lee, D., Nha, K. W., Park, S. J., et al. (2012) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2012.05.881 · Cited by 51
Comparing two methods of identifying alliance rupture events.
Coutinho, J. F. P., Ribeiro, E., Sousa, I., & Safran, J. D. (2013) · Psychotherapy · doi:10.1037/a0032171 · Cited by 63
Halliwick-Based Aquatic Assessments : Reliability and Validity
Tirosh, R., Leurer, M. K., & Getz, M. (2008) · International Journal of Aquatic Research and Education · doi:10.25035/ijare.02.03.04 · Cited by 51
A Quantitative Analysis of Research Publications in Physical Therapy Journals
Miller, P. A., McKibbon, K. A., & Haynes, R. B. (2003) · Physical Therapy · doi:10.1093/ptj/83.2.123 · Cited by 51
Span of control and the significance for public sector managers’ job demands: A multilevel study
Wallin, L., Pousette, A., & Dellve, L. (2013) · Economic and Industrial Democracy · doi:10.1177/0143831x13488002 · Cited by 59
A brief 5-item version of the Neck Disability Index shows good psychometric properties
Walton, D. M., & MacDermid, J. C. (2013) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-11-108 · Cited by 52
Tiffreau, V., Ledoux, I., Eymard, B., et al. (2007) · Neuromuscular Disorders · doi:10.1016/j.nmd.2007.03.014 · Cited by 51
Mehta, L. R., McNeill, M., Hobart, J. C., Wyrwich, K. W., Poon, J. L., Auguste, P., Zhong, J., & Elkins, J. S. (2015) · Multiple Sclerosis Journal - Experimental Translational and Clinical · doi:10.1177/2055217315596993 · Cited by 51
Physical examination tests for the diagnosis of femoroacetabular impingement. A systematic review
Pacheco-Carrillo, A., & Medina-Porqueres, I. (2016) · Physical Therapy in Sport · doi:10.1016/j.ptsp.2016.01.002 · Cited by 54
Accuracies in Measuring Spinopelvic Parameters in Full-Spine Lateral Standing Radiograph
Yamada, K., Aota, Y., Higashi, T., et al. (2015) · Spine · doi:10.1097/brs.0000000000000904 · Cited by 51
Diffusion Tensor Imaging in Pediatric Spinal Cord Injury
Mulcahey, M. J., Samdani, A. F., Gaughan, J., et al. (2011) · Spine · doi:10.1097/brs.0b013e3182470a08 · Cited by 51
Smith, T. W., Uchino, B. N., Florsheim, P., Berg, C. A., Butner, J., Hawkins, M., Henry, N. J. M., Beveridge, R. M., Pearce, G. E., Hopkins, P. N., et al. (2011) · Psychosomatic Medicine · doi:10.1097/psy.0b013e31821188ca · Cited by 51
Mavandadi, V., Bieling, P., & Madsen, V. (2016) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.12310 · Cited by 51
McVeigh, J. A., WINKLER, E. A. H., Healy, G. N., Slater, J., Eastwood, P. R., & Straker, L. M. (2016) · Physiological Measurement · doi:10.1088/0967-3334/37/10/1636 · Cited by 50
Development and Validation of a Photonumeric Scale for Evaluation of Facial Fine Lines
CARRUTHERS, J. D. A., DONOFRIO, L. M., Hardas, B., et al. (2016) · Dermatologic Surgery · doi:10.1097/dss.0000000000000847 · Cited by 51
Albertsen, I. M., Ghédira, M., Graciès, J., et al. (2017) · Journal of Electromyography and Kinesiology · doi:10.1016/j.jelekin.2017.01.005 · Cited by 51
Ultrasound Tomography Evaluation of Breast Density
O'Flynn, E. A. M., Fromageau, J., Ledger, A. E. W., et al. (2017) · Investigative Radiology · doi:10.1097/rli.0000000000000347 · Cited by 51
Assessment of the psychometric properties of the EQ-5D-3L and EQ-5D-5L instruments in psoriasis
Yfantopoulos, J., Chantzaras, A. E., & Kontodimas, S. (2017) · Archives of Dermatological Research · doi:10.1007/s00403-017-1743-2 · Cited by 51
Stirnberg, R., Huijbers, W., Brenner, D., Poser, B. A., Breteler, M. M. B., & Stöcker, T. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.031 · Cited by 51
Predicting group-level outcome variables: An empirical comparison of analysis strategies
Foster‐Johnson, L., & Kromrey, J. D. (2018) · Behavior Research Methods · doi:10.3758/s13428-018-1025-8 · Cited by 51
Harris‐Hayes, M., Steger-May, K., Dillen, L. R. V., Schootman, M., Salsich, G. B., Czuppon, S., Clohisy, J. C. F., Commean, P. K., Hillen, T. J., Sahrmann, S. A., et al. (2018) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2018.7810 · Cited by 56
Kinect-based assessment of proximal arm non-use after a stroke
Bakhti, K., Laffont, I., Muthalib, M., Froger, J., & Mottet, D. (2018) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-018-0451-2 · Cited by 52
Nafees, B., Størling, Z. M., Hindsberger, C., et al. (2018) · Health and Quality of Life Outcomes · doi:10.1186/s12955-018-1054-0 · Cited by 51
Ottenstein, C., & Lischetzke, T. (2019) · Assessment · doi:10.1177/1073191119839138 · Cited by 65
Suppliers versus start-ups: Where do better innovation ideas come from?
Homfeldt, F., Rese, A., & Simon, F. (2019) · Research Policy · doi:10.1016/j.respol.2019.04.002 · Cited by 51
Provoost, S., Ruwaard, J., van Breda, W., Riper, H. M., & Bosse, T. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01065 · Cited by 54
Wu, J. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.104066 · Cited by 52
Assessing Raters: What Factors Predict Discernment in Novice Creativity Raters?
Ceh, S. M., Edelmann, C., Hofer, G., & Benedek, M. (2021) · The Journal of Creative Behavior · doi:10.1002/jocb.515 · Cited by 50
Phenotypic diversity in white adults with moderate to severe Class II malocclusion
Uribe, L. M. M., Howe, S. C., Kummet, C. M., et al. (2014) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2013.11.013 · Cited by 51
Luche, C. D., Durrant, S., Poltrock, S., & Floccia, C. (2015) · Infant Behavior and Development · doi:10.1016/j.infbeh.2015.05.005 · Cited by 51
Desire for Hastened Death Among Patients With Advanced AIDS
Rosenfeld, B. D., BREITBART, W. S., Gibson, C., Kramer, M., Tomarken, A., Nelson, C. J., Pessin, H., Funesti-Esch, J., Galietta, M., Garcia, N., et al. (2006) · Psychosomatics · doi:10.1176/appi.psy.47.6.504 · Cited by 51
Maldonado, M. J., Nieto, J. C., Díez-Cuenca, M., & Pinero, D. P. (2006) · Ophthalmology · doi:10.1016/j.ophtha.2006.05.053 · Cited by 52
Video based analysis of dynamic midfoot function and its relationship with Foot Posture Index scores
Nielsen, R. O., Rathleff, M. S., Moelgaard, C. M., Simonsen, O. H., Kaalund, S., Olesen, C. G., Christensen, F. B., & Kersting, U. G. (2009) · Gait & Posture · doi:10.1016/j.gaitpost.2009.09.012 · Cited by 50
Gross, M. T., Clemence, L. M., Cox, B. D., et al. (1997) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.1997.25.4.245 · Cited by 51
Farmer, C., & Aman, M. G. (2009) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2009.09.014 · Cited by 51
Luiz, R. R., & Szklo, M. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.07.007 · Cited by 51
Determinants of Disabling Foot Pain in Retirement Village Residents
Menz, H. B., & Morris, M. E. (2005) · Journal of the American Podiatric Medical Association · doi:10.7547/0950573 · Cited by 51
Fleming, A., Cook, K. F., Nelson, N. D., & Lai, E. C. (2005) · Movement Disorders · doi:10.1002/mds.20592 · Cited by 51
Accuracy and Reliability of Maternal Recall of Infant Birth Weight Among Older Women
Catov, J. M., Newman, A. B., Kelsey, S. F., Roberts, J. M., Sutton-Tyrrell, K., Garcia, M., Ayonayon, H. N., Tylavsky, F., & Ness, R. B. (2005) · Annals of Epidemiology · doi:10.1016/j.annepidem.2005.09.004 · Cited by 51
Is the Revised Psychopathy Checklist Clinically Useful for Adolescents?
Ridenour, T. A., Marchant, G. J., & Dean, R. S. (2001) · Journal of Psychoeducational Assessment · doi:10.1177/073428290101900303 · Cited by 51
KRUPNICK, J. L., Elkin, I., Collins, J., Simmens, S. J., Sotsky, S. M., Pilkonis, P. A., & Watkins, J. T. (1994) · Psychotherapy · doi:10.1037/0033-3204.31.1.28 · Cited by 50
Anhedonia and emotional responses to affect evoking stimuli
Berenbaum, H., Snowhite, R., & Oltmanns, T. F. (1987) · Psychological Medicine · doi:10.1017/s0033291700025915 · Cited by 51
Lyden, P. D., Zweifler, R. M., Mahdavi, Z., & Lonzo, L. (1994) · Stroke · doi:10.1161/01.str.25.12.2421 · Cited by 51
Pathoclysis in aging human cerebral cortex: Evidence from in vivo MRI morphometry
Raz, N., Torres, I. J., Spencer, W. D., & Acker, J. D. (1993) · Psychobiology · doi:10.3758/bf03332042 · Cited by 51
Portable Sleep Monitoring for Diagnosing Sleep Apnea in Hospitalized Patients With Heart Failure
Aurora, R. N., Patil, S. P., & Punjabi, N. M. (2018) · Chest · doi:10.1016/j.chest.2018.04.008 · Cited by 50
Florsheim, P., McArthur, L., Hudak, C., Heavin, S., & Burrow-Sanchez, J. (2011) · Journal of Couple & Relationship Therapy · doi:10.1080/15332691.2011.562823 · Cited by 49
The role of personal goals in autonoetic experience when imagining future events
Lehner, E., & D’Argembeau, A. (2016) · Consciousness and Cognition · doi:10.1016/j.concog.2016.04.002 · Cited by 49
Youdas, J. W., Hollman, J. H., & Krause, D. A. (2006) · Physiotherapy Theory and Practice · doi:10.1080/09593980600927864 · Cited by 75
Spontaneous facial expression in unscripted social interactions can be measured automatically
Girard, J. M., Cohn, J. F., Jeni, L. A., Sayette, M. A., & De la Torre, F. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0536-1 · Cited by 69
Nielsen, M., Mushin, I., Tomaselli, K. G., & Whiten, A. (2016) · Child Development · doi:10.1111/cdev.12504 · Cited by 54
Reisinger, D. L., Shaffer, R. C., Horn, P. S., Hong, M. P., Pedapati, E. V., Dominick, K. C., & Erickson, C. A. (2020) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2019.00076 · Cited by 51
Association of acute psychosocial stress with oxidative stress: Evidence from serum analysis
Kim, E., Zhao, Z., Rzasa, J. R., Glassman, M., Bentley, W. E., Chen, S., Kelly, D. L., & Payne, G. F. (2021) · Redox Biology · doi:10.1016/j.redox.2021.102138 · Cited by 54
Household food waste and theory of planned behavior: A systematic review and meta-analysis
Srivastava, S. K., Mishra, A., Singh, S., & Jaiswal, D. (2023) · Environmental Science and Pollution Research · doi:10.1007/s11356-023-29141-0 · Cited by 59
Zhang, S., Zhou, H., Tchantchane, R., & Alıcı, G. (2024) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2024.3386929 · Cited by 50
The reliability of weight‐for‐length/height Z scores in children
Mwangome, M., & Berkley, J. A. (2014) · Maternal and Child Nutrition · doi:10.1111/mcn.12124 · Cited by 50
Weier, K., Fonov, V., Lavoie, K., Doyon, J., & Collins, D. L. (2014) · Human Brain Mapping · doi:10.1002/hbm.22529 · Cited by 49
Padilla‐Walker, L. M., Coyne, S. M., Collier, K. M., & Nielson, M. G. (2015) · Developmental Psychology · doi:10.1037/a0039488 · Cited by 50
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
