The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 2,801–2,900 of 23,883
Nentwich, M., Ai, L., Madsen, J., Telesford, Q. K., Haufe, S., Milham, M. P., & Parra, L. C. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117001 · Cited by 104
Higher-order organization of multivariate time series
Santoro, A., Battiston, F., Petri, G., & Amico, E. (2023) · Nature Physics · doi:10.1038/s41567-022-01852-0 · Cited by 106
CT Angiography of the Aorta Using Photon-counting Detector CT with Reduced Contrast Media Volume
Higashigaito, K., Mergen, V., Eberhard, M., Jungblut, L., Hebeisen, M., Rätzer, S., Zanini, B., Kobe, A. R., Martini, K., Euler, A., et al. (2023) · Radiology Cardiothoracic Imaging · doi:10.1148/ryct.220140 · Cited by 104
Effectiveness of targeting the vulnerability factors of depression in cognitive therapy.
Hayes, A. M., Castonguay, L. G., & Goldfried, M. R. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.3.623 · Cited by 102
Uswatte, G., & Taub, E. (2005) · Rehabilitation Psychology · doi:10.1037/0090-5550.50.1.34 · Cited by 100
Houtte, M. V., & Stevens, P. A. J. (2008) · Youth & Society · doi:10.1177/0044118x08316251 · Cited by 111
I know what you did: The effects of interpersonal deviance on bystanders.
Ferguson, M. J., & Barry, B. (2011) · Journal of Occupational Health Psychology · doi:10.1037/a0021708 · Cited by 103
Miele, G. M., Carpenter, K. M., Cockerham, M. S., Trautman, K. D., BLAINE, J. D., & Hasin, D. S. (2000) · Drug and Alcohol Dependence · doi:10.1016/s0376-8716(99)00111-8 · Cited by 102
Verhagen, A. P., de Vet, H. C. W., de Bie, R. A., Kessels, A. G. H., Boers, M., & Knipschild, P. G. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00297-7 · Cited by 102
Vaskinn, A., Sundet, K. S., Simonsen, C., Hellvin, T., Melle, I. S., & Andreassen, O. A. (2011) · Neuropsychology · doi:10.1037/a0022677 · Cited by 107
Schudlich, T. D. R., Papp, L. M., & Cummings, E. M. (2011) · Journal of Family Psychology · doi:10.1037/a0024216 · Cited by 102
Smits-Engelsman, B. C. M., Niemeijer, A. S., & Waelvelde, H. V. (2011) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2011.01.031 · Cited by 102
Hinton-Bayre, A. D., & Geffen, G. M. (2005) · Psychological Assessment · doi:10.1037/1040-3590.17.2.237 · Cited by 102
Furr-Holden, D., Smart, M. J., Pokorni, J. L., et al. (2008) · Prevention Science · doi:10.1007/s11121-008-0107-8 · Cited by 102
Li, Z., Kadivar, A., Pluta, J. B., Dunlop, J., & Wang, Z. (2012) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.23670 · Cited by 103
Radiographic correlates of hallux valgus severity in older people
D'Arcangelo, P. R., Landorf, K. B., Munteanu, S. E., Zammit, G. V., & Menz, H. B. (2010) · Journal of Foot and Ankle Research · doi:10.1186/1757-1146-3-20 · Cited by 101
Aas, M., Haukvik, U. K., Djurovic, S., Bergmann, Ø., Athanasiu, L., Tesli, M., Hellvin, T., Steen, N. E., Agartz, I., Lorentzen, S., et al. (2013) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2013.07.008 · Cited by 104
The Uniqueness of the Human Anterior Dentition: A Geometric Morphometric Analysis
Kieser, J. A., Bernal, V., Waddell, J. N., et al. (2007) · Journal of Forensic Sciences · doi:10.1111/j.1556-4029.2007.00403.x · Cited by 102
Reciprocal influences between parenting and adolescent problem-solving behavior.
Rueter, M. A., & Conger, R. D. (1998) · Developmental Psychology · doi:10.1037//0012-1649.34.6.1470 · Cited by 102
Sclan, S. G., SAILLON, A., Franssen, E., Hugonot-Diener, L., Saillon, A., & Reisberg, . (1996) · International Journal of Geriatric Psychiatry · doi:10.1002/(sici)1099-1166(199609)11:9 · Cited by 102
Rentrop, K. P., Feit, F., Sherman, W., et al. (1989) · Journal of the American College of Cardiology · doi:10.1016/0735-1097(89)90054-5 · Cited by 102
Maternal smoking during pregnancy and regional brain volumes in preterm infants
Ekblad, M., Korkeila, J., Parkkola, R., Lapinleimu, H., Haataja, L., & Lehtonen, L. (2010) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2009.07.061 · Cited by 100
Pedersen, L., Kunz, C., Rasmussen, K. B., et al. (2010) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2010.526681 · Cited by 101
Connelly, S., & Ruark, G. (2010) · The Leadership Quarterly · doi:10.1016/j.leaqua.2010.07.005 · Cited by 101
Age-Related Differences in Test-Retest Reliability in Resting-State Brain Functional Connectivity
Song, J., Desphande, A. S., Meier, T. B., Tudorascu, D., Vergun, S., Nair, V. A., Biswal, B. B., Meyerand, M. E., Birn, R. M., Bellec, P., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0049847 · Cited by 104
Seitz, A. L., McClure, P. W., Lynch, S., Ketchum, J. M., & Michener, L. A. (2011) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2011.01.008 · Cited by 100
Ronald, A., Viding, E. M., Happe, F. G. E., & Plomin, R. J. (2006) · Social Neuroscience · doi:10.1080/17470910601068088 · Cited by 103
Christensen, T. B., Eikenæs, I. U.-M., Hummelen, B., Pedersen, G., Nysæter, T. E., Bender, D. S., Skodol, A. E., & Selvik, S. G. (2019) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000352 · Cited by 102
Wiebe, D. J., Storey, E. P., Orchinik, J. E., Grady, M. F., Leddy, J. J., Willer, B. S., Haider, M. N., Mannix, R., Meehan, W. P., Vernau, B. T., et al. (2021) · Clinical Journal of Sport Medicine · doi:10.1097/jsm.0000000000000946 · Cited by 115
Brief segments of neurophysiological activity enable individual differentiation
da Silva Castanheira, J., Orozco Perez, H. D., Misic, B., & Baillet, S. (2021) · Nature Communications · doi:10.1038/s41467-021-25895-8 · Cited by 101
Variation of semen measures within normal men
Poland, M. L., Moghissi, K. S., Giblin, P. T., et al. (1985) · Fertility and Sterility · doi:10.1016/s0015-0282(16)48866-7 · Cited by 101
A Simulator-Based Tool That Assesses Pediatric Resident Resuscitation Competency
Brett-Fleegler, M. B., Vinci, R. J., Weiner, D. L., Harris, S. K., Shih, M., & Kleinman, M. E. (2008) · PEDIATRICS · doi:10.1542/peds.2005-1259 · Cited by 101
Dang, N. R., Moreau, M. J., Hill, D. L., et al. (2005) · Spine · doi:10.1097/01.brs.0000160840.51621.6b · Cited by 101
Långström, N. (2002) · Psychology Crime and Law · doi:10.1080/10683160208401808 · Cited by 101
Family interactions in adoptive compared to nonadoptive families.
Rueter, M. A., Keyes, M. A., Iacono, W. G., & McGue, M. K. (2009) · Journal of Family Psychology · doi:10.1037/a0014091 · Cited by 115
Dumas, H. M., Fragala-Pinkham, M. A., Haley, S. M., Ni, P., Coster, W. J., Kramer, J. M., Kao, Y., Moed, R., & Ludlow, L. H. (2011) · Disability and Rehabilitation · doi:10.3109/09638288.2011.607217 · Cited by 102
Test-Retest reliability of Biodex Balance SD on physically active old people
Parraça, J. A., Olivares, P. R., Carbonell‐Baeza, A., Aparicio, V. A., Adsuar, J. C., & Gusi, N. (2011) · Journal of Human Sport and Exercise · doi:10.4100/jhse.2011.62.25 · Cited by 102
Aggression at Age 5 as a Function of Prenatal Exposure to Cocaine, Gender, and Environmental Risk
Bendersky, M., Bennett, D. S., & Lewis, M. (2005) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsj025 · Cited by 101
Correlates of Coparenting During Infancy*
Lindsey, E. W., Caldera, Y. M., & Colwell, M. J. (2005) · Family Relations · doi:10.1111/j.1741-3729.2005.00322.x · Cited by 101
Rybicki, B. A., Neslund-Dudas, C. M., Nock, N. L., et al. (2006) · Cancer Detection and Prevention · doi:10.1016/j.cdp.2006.09.004 · Cited by 101
Elsäßer, M., & Wirtz, B. W. (2017) · Journal of Business and Industrial Marketing · doi:10.1108/jbim-05-2015-0101 · Cited by 101
Niemann, K., Hammers, A., Coenen, V. A., Thron, A. K., & Klosterkötter, J. (2000) · Psychiatry Research Neuroimaging · doi:10.1016/s0925-4927(00)00059-7 · Cited by 101
Winhusen, T., Kropp, F., Babcock, D., Hague, D., Erickson, S. J., Renz, C., Rau, L., Lewis, D., Leimberger, J., & Somoza, E. (2008) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2007.09.006 · Cited by 99
Britton, J. C., Bar-Haim, Y., Clementi, M. A., Sankin, L. S., Chen, G., Shechner, T., Norcross, M. A., Spiro, C. N., Lindstrom, K. M., & Pine, D. S. (2013) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2012.11.001 · Cited by 101
fMRI reliability: Influences of task and experimental design
Bennett, C. M., & Miller, M. B. (2013) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-013-0195-1 · Cited by 100
Biabani, M., Farrell, M. J., Zoghi, M., et al. (2018) · Neuroscience Letters · doi:10.1016/j.neulet.2018.03.026 · Cited by 100
Subthalamic Nucleus Deep Brain Stimulation Modulates 2 Distinct Neurocircuits
Shen, L., Jiang, C., Hubbard, C. S., Ren, J., He, C., Wang, D., Dahmani, L., Guo, Y., Liu, Y., Xu, S., et al. (2020) · Annals of Neurology · doi:10.1002/ana.25906 · Cited by 100
Do They Want to Cross? Understanding Pedestrian Intention for Behavior Prediction
Kotseruba, I., Rasouli, A., & Tsotsos, J. K. (2020) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv47402.2020.9304591 · Cited by 100
Radiomics feature robustness as measured using an MRI phantom
Lee, J., Steinmann, A., Ding, Y., Lee, H. J., Owens, C. A., Wang, J., Yang, J., Followill, D. S., Ger, R. B., Mackin, D., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-83593-3 · Cited by 100
Kolber, M. J., Pizzini, M., Robinson, A., et al. (2013) · PubMed · Cited by 100
Potthast, S., Mitsumori, L. M., Stanescu, L. A., Richardson, M. L., Branch, K. R., Dubinsky, T. J., & Maki, J. H. (2009) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.22016 · Cited by 99
Shinohara, Y., Minematsu, K., Amano, T., & Ohashi, Y. (2006) · Cerebrovascular Diseases · doi:10.1159/000091226 · Cited by 101
Falls classification using tri-axial accelerometers during the five-times-sit-to-stand test
Doheny, E. P., Walsh, C., Foran, T. G., Greene, B. R., Fan, C. W., Cunningham, C., & Kenny, R. A. (2013) · Gait & Posture · doi:10.1016/j.gaitpost.2013.05.013 · Cited by 100
Arthroscopic screw fixation of large anterior glenoid fractures
Tauber, M., Moursy, M., Eppel, M., et al. (2007) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-007-0437-2 · Cited by 100
Consequences of increased neuromotor noise for reaching movements in persons with stroke
McCrea, P. H., & Eng, J. J. (2004) · Experimental Brain Research · doi:10.1007/s00221-004-2106-8 · Cited by 100
Santos, C., Bernardes, J., Ayres‐de‐Campos, D., Costa, A., & Amorim‐Costa, C. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.11.010 · Cited by 99
Feeny, D., Furlong, W., Saigal, S., & Sun, J. (2004) · Social Science & Medicine (1967) · doi:10.1016/s0277-9536(03)00254-5 · Cited by 106
Periodontal attachment loss attributable to cigarette smoking in an urban Brazilian population
Susin, C., Oppermann, R. V., Haugejorden, O., & Albandar, J. M. (2004) · Journal Of Clinical Periodontology · doi:10.1111/j.1600-051x.2004.00588.x · Cited by 97
Morris, J. H., van Wijck, F., Joice, S. A., Ogston, S. A., Cole, I., & MacWalter, R. S. (2008) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2007.11.039 · Cited by 100
A new method to estimate adult age‐at‐death using the acetabulum
Calce, S. E. (2012) · American Journal of Physical Anthropology · doi:10.1002/ajpa.22026 · Cited by 100
An objective device for measuring surface roughness of skin and scars
Bloemen, M. C. T., van Gerven, M. S., van der Wal, M. B. A., et al. (2011) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2010.03.006 · Cited by 100
Cottone, P., Sabino, V., Steardo, L., & Zorrilla, E. P. (2008) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2008.08.010 · Cited by 104
Jenkins, N. D. M., Housh, T. J., Buckner, S. L., Bergstrom, H. C., Cochrane, K. C., Hill, E. C., Smith, C. M., Schmidt, R. J., Johnson, G. O., & Cramer, J. T. (2015) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000001308 · Cited by 110
Repeatability of vascular responsiveness measures derived from near-infrared spectroscopy
McLay, K. M., Nederveen, J. P., Pogliaghi, S., Paterson, D. H., & Murias, J. M. (2016) · Physiological Reports · doi:10.14814/phy2.12772 · Cited by 106
Genova, J. M., & Gross, M. T. (2000) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2000.30.11.664 · Cited by 100
Brain changes in schizophrenia
Sharma, T., Lancaster, E., Lee, D., et al. (1998) · The British Journal of Psychiatry · doi:10.1192/bjp.173.2.132 · Cited by 100
Boris, N. W., Zeanah, C. H., Larrieu, J. A., Scheeringa, M. S., & Heller, S. S. (1998) · American Journal of Psychiatry · doi:10.1176/ajp.155.2.295 · Cited by 99
Relations of Balance Function and Gross Motor Ability for Children with Cerebral Palsy
Liao, H.-F., & Hwang, A.-W. (2003) · Perceptual and Motor Skills · doi:10.2466/pms.2003.96.3c.1173 · Cited by 97
Effects of treadmill running and fatigue on impact acceleration in distance running
Pérez, J. A. G., Pérez‐Soriano, P., Belloch, S. L., Lucas‐Cuevas, A. G., & Zuriaga, D. S. (2014) · Sports Biomechanics · doi:10.1080/14763141.2014.909527 · Cited by 99
Task Rotation and Polychronicity: Effects on Individuals' Creativity
Madjar, N. Y., & Oldham, G. R. (2006) · Human Performance · doi:10.1207/s15327043hup1902_2 · Cited by 99
Stability of low-frequency fluctuation amplitudes in prolonged resting-state fMRI
Küblböck, M., Woletz, M., Höflich, A. S., Sladky, R., Kranz, G. S., Hoffmann, A., Lanzenberger, R. R., & Windischberger, C. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.09.038 · Cited by 100
A Mixed-Methods Investigation of Instructional Coaching for Teachers of Diverse Learners
Teemant, A. (2013) · Urban Education · doi:10.1177/0042085913481362 · Cited by 120
Facial Indicators of Positive Emotions in Rats
Finlayson, K., Lampe, J. F., Hintze, S., Würbel, H., & Melotti, L. (2016) · PLoS ONE · doi:10.1371/journal.pone.0166446 · Cited by 103
Britt, T. W., Shuffler, M. L., Pegram, R., Xoxakos, P., Rosopa, P. J., Hirsh, E., & Jackson, W. (2020) · Applied Psychology · doi:10.1111/apps.12304 · Cited by 119
Casula, E. P., Pellicciari, M. C., Bonnı̀, S., Spanò, B., Ponzo, V., Salsano, I., Giulietti, G., Cinnera, A. M., Maiella, M., Borghi, I., et al. (2021) · Human Brain Mapping · doi:10.1002/hbm.25297 · Cited by 99
Patellofemoral Alignment: Reliability
Tomsich, D. A., Nitz, A. J., Threlkeld, A. J., et al. (1996) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.1996.23.3.200 · Cited by 99
Silfies, S. P., Bhattacharya, A., Biely, S. A., Smith, S. S., & Giszter, S. F. (2008) · Gait & Posture · doi:10.1016/j.gaitpost.2008.10.053 · Cited by 98
Yoong, S. L., Carey, M. L., D’Este, C. A., & Sanson‐Fisher, R. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-38 · Cited by 100
Reproducibility of maximal exercise test data in the HERITAGE Family Study
Skinner, J. S., WILMORE, K. M., Jaskólska, A., Jaskólski, A., Daw, E. W., Rice, T., Gagnon, J., Leon, A. S., Wilmore, J. H., Rao, D. C., et al. (1999) · Medicine & Science in Sports & Exercise · doi:10.1097/00005768-199911000-00020 · Cited by 99
Attachment style and relationship satisfaction: Test of a self-disclosure explanation.
Keelan, J. P. R., Dion, K. K., & Dion, K. L. (1998) · Canadian Journal of Behavioural Science/Revue canadienne des sciences du comportement · doi:10.1037/h0087055 · Cited by 99
Leitner, C., Mair, P., Paul, B., Wick, F., Mittermaier, C., Sycha, T., & EBENBICHLER, G. R. (2007) · Journal of Electromyography and Kinesiology · doi:10.1016/j.jelekin.2007.09.007 · Cited by 98
Stability of the diurnal cortisol profile in children and adolescents
Rotenberg, S., McGrath, J. J., Roy‐Gagnon, M., & Tu, M. T. (2012) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2012.04.014 · Cited by 112
Bird, H., CANINO, G. J., Davies, M., Ramírez, R., Chavez, L. M., Duarte, C. S., & Shen, S. (2005) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000163281.41383.94 · Cited by 99
Minimum Detectable Change in Gait Velocity during Acute Rehabilitation following Hip Fracture
Hollman, J. H., Beckman, B. A., Brandt, R. A., Merriwether, E. N., Williams, R. T., & Nordrum, J. T. (2008) · Journal of Geriatric Physical Therapy · doi:10.1519/00139143-200831020-00003 · Cited by 99
Bellavia, D., Michelena, H., Martinez, M., Pellikka, P. A., Bruce, C. J., Connolly, H. M., Villarraga, H. R., Veress, G., Oh, J. K., & Miller, F. A. (2009) · Heart · doi:10.1136/hrt.2009.182170 · Cited by 99
Chou, Y., Panych, L. P., Dickey, C. C., Petrella, J. R., & Chen, N. (2012) · American Journal of Neuroradiology · doi:10.3174/ajnr.a2894 · Cited by 113
Actigraphy Scoring Reliability in the Study of Osteoporotic Fractures
Blackwell, T., Ancoli‐Israel, S., Gehrman, P., Schneider, J. L., Pedula, K. L., & Stone, K. L. (2005) · SLEEP · doi:10.1093/sleep/28.12.1599 · Cited by 99
Grancharov, V., & Kleijn, W. B. (2007) · Springer handbooks · doi:10.1007/978-3-540-49127-9_5 · Cited by 99
McHorney, C. A., Rust, J. N., Golombok, S. E., Davis, S. R., Bouchard, C., Brown, C., Basson, R. J., Sarti, C. D., Kuznicki, J. T., Rodenberg, C. A., et al. (2004) · Menopause The Journal of The North American Menopause Society · doi:10.1097/01.gme.0000109316.11228.77 · Cited by 99
Tang, D. L., Cho, S., Rundle, A., Chen, S., Phillips, D. H., Zhou, J., Hsu, Y., Schnabel, F. R., Estabrook, A., & Perera, F. P. (2002) · Breast Cancer Research and Treatment · doi:10.1023/a:1019693504183 · Cited by 99
Makris, N., Schlerf, J. E., Hodge, S. M., Haselgrove, C., Albaugh, M. D., Seidman, L. J., Rauch, S. L., Harris, G. J., Biederman, J., Caviness, V. S., et al. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.056 · Cited by 99
Can Prediction of Violence Among Psychiatric Inpatients Be Improved?
Blomhoff, S., Seim, S., & Friis, S. (1990) · Psychiatric Services · doi:10.1176/ps.41.7.771 · Cited by 99
A meta-analysis of vibrotactile and visual information displays for improving task performance
Prewett, M. S., Elliott, L. R., Walvoord, A. G., & Coovert, M. D. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2010.2103057 · Cited by 99
Psychometric properties of the EQ-5D in a study of people with mild to moderate dementia
Kunz, S. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9600-1 · Cited by 98
Reliability and agreement studies: a guide for clinical investigators
Hernáez, R. (2015) · Gut · doi:10.1136/gutjnl-2014-308619 · Cited by 98
Engelken, F., Wassilew, G. I., Köhlitz, T., Brockhaus, S., Hamm, B., Perka, C., & Diederichs, G. (2013) · The Journal of Arthroplasty · doi:10.1016/j.arth.2013.04.045 · Cited by 98
Hu, Z., Fang, X., Zhou, Z., et al. (2013) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.l.01607 · Cited by 98
Interrater and Intrarater Reliability of the Active Knee Extension (AKE) Test among Healthy Adults
Hamid, M. S. A., Ali, M. R., & Yusof, A. (2013) · Journal of Physical Therapy Science · doi:10.1589/jpts.25.957 · Cited by 102
Zimmermann, J., Benecke, C., Bender, D. S., Skodol, A. E., Schauenburg, H., Cierpka, M., & Leising, D. (2013) · Journal of Personality Assessment · doi:10.1080/00223891.2013.852563 · Cited by 98
Reznicek, L., Muth, D. R., Kampik, A., et al. (2013) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2013-303400 · Cited by 98
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
