The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 16,201–16,300 of 23,882
Xu, Y., & Stocco, A. (2021) · Computational Brain & Behavior · doi:10.1007/s42113-021-00102-5 · Cited by 6
Jacquot, A., Gauci, M.-O., Urvoy, M., et al. (2021) · Shoulder & Elbow · doi:10.1177/17585732211001756 · Cited by 6
The Italian Validation of OSCI: The Organizational and Safety Climate Inventory
Platania, S., Paolillo, A., & da Silva, S. A. (2021) · Safety · doi:10.3390/safety7010022 · Cited by 6
Circumference Method Estimates Percent Body Fat in Male US Service Members with Lower Limb Loss
George, B., Pruziner, A. L., & Andrews, A. M. (2021) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2021.02.009 · Cited by 6
López, F. A., Faraone, S. V., Newcorn, J. H., Doll, H., Rhoten, S. E., Lewis, H. B., Khan, T. F., DeSousa, N. J., Sallee, F. R., Incledon, B., et al. (2021) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2020.0159 · Cited by 6
The flexible impact of member affect in groups performing complex decision-making tasks
Yoon, Y., Larson, J. R., & Huntsinger, J. R. (2021) · Group Processes & Intergroup Relations · doi:10.1177/1368430220985601 · Cited by 6
Fostering Computational Thinking Through Data Visualization and Design on Secondary School Students
Özkök, G. A. (2021) · JUCS - Journal of Universal Computer Science · doi:10.3897/jucs.66265 · Cited by 6
Protocol for a scoping review exploring the use of patient-reported outcomes in adult social care
Hughes, S., Aiyegbusi, O. L., Lasserson, D. S., Collis, P., Rivera, S. C., McMullan, C., Turner, G., Glasby, J., & Calvert, M. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045206 · Cited by 9
Horsak, B., Schwab, C., Durstberger, S., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-89763-7 · Cited by 6
Kocahan, T., Akınoğlu, B., Yılmaz, A. E., Rosemann, T. J., & Knechtle, B. (2021) · Applied Sciences · doi:10.3390/app11114951 · Cited by 6
Deep learning for semi-automated unidirectional measurement of lung tumor size in CT
Woo, M., Devane, A. M., Lowe, S. C., Lowther, E. L., & Gimbel, R. W. (2021) · Cancer Imaging · doi:10.1186/s40644-021-00413-7 · Cited by 6
Braarud, P. Ø. (2021) · Applied Ergonomics · doi:10.1016/j.apergo.2021.103500 · Cited by 6
Calthorpe, S., Kimmel, L. A., Fitzgerald, M., et al. (2021) · Physical Therapy · doi:10.1093/ptj/pzab183 · Cited by 6
Deres-Cohen, K., Lipsitz‐Odess, I., Fisher, H., Ramseyer, F., Lutz, W., & Zilcha‐Mano, S. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1966542 · Cited by 7
Miller, M. L., Ghisletta, P., Jacobs, B. S., Dahle, C. L., & Raz, N. (2021) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2021.08.014 · Cited by 7
Abdullah, T., Ali, A., Saka, E., Canbaz, M., Gökduman, H. C., Polat, Ö., Esen, F., & Sungur, M. O. (2021) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-021-00752-7 · Cited by 7
The Neonatal Assessment Manual scorE: A Reliability Study on Hospitalized Neonates
Manzotti, A., Cerritelli, F., Lombardi, E., et al. (2021) · Frontiers in Pediatrics · doi:10.3389/fped.2021.715091 · Cited by 6
Connecting Teacher Training to Task-Based Language Teaching Implementation
Bryfonski, L. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108868327.027 · Cited by 7
Child Interaction in Task-Supported EFL/CLIL Contexts
Mayo, M. D. P. G. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108868327.023 · Cited by 6
Lipsitz‐Odess, I., Benisty, H., Dolev‐Amit, T., & Zilcha‐Mano, S. (2021) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2700 · Cited by 6
Li, Q., Liao, X., Huang, X. M., et al. (2022) · International Journal of Occupational Safety and Ergonomics · doi:10.1080/10803548.2022.2029016 · Cited by 6
Li, Y., Chen, X., Ma, X., & Lu, X. (2022) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2022.01.007 · Cited by 6
Logan, A. A., Nielsen, B. D., Hallock, D. B., et al. (2021) · Journal of Equine Veterinary Science · doi:10.1016/j.jevs.2021.103862 · Cited by 6
Sun, C., & Lu, J. J. (2022) · Discrete Dynamics in Nature and Society · doi:10.1155/2022/7049765 · Cited by 6
Do Fitter Children Better Assess Their Physical Activity with Questionnaire Than Less Fit Children?
Premelč, J., Meh, K., Vähä‐Ypyä, H., Sember, V., & Jurak, G. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19031304 · Cited by 9
Brighi, C., Puttick, S. G., Li, S., et al. (2022) · EJNMMI Physics · doi:10.1186/s40658-022-00438-2 · Cited by 6
Williams, P. A., Burke, T. A., Norquist, J. M., et al. (2022) · JTO Clinical and Research Reports · doi:10.1016/j.jtocrr.2022.100298 · Cited by 6
Fieseler, G., Laudner, K. G., Sendler, J., et al. (2022) · JSES International · doi:10.1016/j.jseint.2022.01.011 · Cited by 6
Strony, J. T., LaBelle, M. W., McMellen, C. J., et al. (2022) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2022.04.013 · Cited by 6
Can People Intentionally and Selectively Forget Prose Material?
Pastötter, B., & Haciahmet, C. C. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.928533 · Cited by 5
Lower Extremity Somatosensory Function Throughout Concussion Recovery: A Prospective Cohort Study
Lempke, L. B., Hoch, M. C., Call, J. A., et al. (2022) · Journal of Head Trauma Rehabilitation · doi:10.1097/htr.0000000000000805 · Cited by 6
Brighi, C., Verburg, N., Koh, E., Walker, A., Chen, C., Pillay, S., Hamer, P. C. D. W., Aly, F., Holloway, L. C., Keall, P., et al. (2022) · Physics and Imaging in Radiation Oncology · doi:10.1016/j.phro.2022.06.004 · Cited by 6
Wang, A. Y.-D., Kanter, M. J., Olmos, M., et al. (2022) · Journal of Neurosurgery Spine · doi:10.3171/2022.5.spine22362 · Cited by 6
Biberdzic, M., & Grenyer, B. F. S. (2022) · Journal of Personality Assessment · doi:10.1080/00223891.2022.2086133 · Cited by 9
Vetter, N. C., Fröhner, J. H., Hoffmann, K., Backhausen, L. L., & Smolka, M. N. (2022) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2022.101131 · Cited by 6
Midwives’ ability during third stage of childbirth to estimate postpartum haemorrhage
Wiklund, I. E., Fernández, S. A., & Jonsson, M. (2022) · European Journal of Obstetrics & Gynecology and Reproductive Biology X · doi:10.1016/j.eurox.2022.100158 · Cited by 6
Lehra, C., Omalekb, O., Osborne, S., et al. (2022) · International Journal of Kinesiology and Sports Science · doi:10.7575/aiac.ijkss.v.10n.2p.1 · Cited by 6
Lazarus, G., Pshedetzky‐Shochat, R., Zahavi, T., Gleason, M. E. J., & Rafaeli, E. (2021) · Journal of Social and Personal Relationships · doi:10.1177/02654075211043291 · Cited by 8
Paik, Y., & Lee, E. (2022) · Journal of Leadership & Organizational Studies · doi:10.1177/15480518221118450 · Cited by 6
24-2 SITA Standard versus 24-2 SITA Faster in Perimetry-Naive Normal Subjects
Costa, V. P., Zangalli, C. E. S., Jammal, A. A., Medeiros, F. A., Miyazaki, J. V. M. K., Perez, V., Boscarioli, M. L. N., & Schimiti, R. B. (2022) · Ophthalmology Glaucoma · doi:10.1016/j.ogla.2022.08.006 · Cited by 6
Papagiannopoulou, E., Laiou, E., Tatsi, C., Dimakopoulos, G., Ntzani, E., Siamopoulos, K. C., & Tatsioni, A. (2022) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13759 · Cited by 6
Abebe, A. B., Ayele, T. A., & Miller, J. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05833-5 · Cited by 6
Detecting Illegal Intercountry Trade of Mercury Using Discrepancies in Mirrored Trade Data
Fuse, M., Oda, H., Noguchi, H., et al. (2022) · Environmental Science & Technology · doi:10.1021/acs.est.2c04327 · Cited by 6
Hill, K. E., Haney, A. M., Foti, D., Aslinger, E. N., Thomas, K. M., & Lane, S. P. (2022) · Psychophysiology · doi:10.1111/psyp.14185 · Cited by 6
Treatment Fidelity and Outcome in CBT for Youth with Autism: The MEYA Fidelity Scale
McLeod, B. D., Wood, J. J., Cecilione, J. L., Dunn, D., Kendall, P. C., Storch, E. A., Cho, A. C., & Rosenau, K. A. (2022) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2022.2124515 · Cited by 7
Larsen, J. A. (2022) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2022.101147 · Cited by 6
Min, S., Shin, H. D., Kim, Y. K., et al. (2022) · Injury · doi:10.1016/j.injury.2022.10.004 · Cited by 6
Dam (Canis familiaris) Welfare throughout the Peri-Parturient Period in Commercial Breeding Kennels
Romaniuk, A. C., Barnard, S., Weller, J. E., Weng, H., Dangoudoubiyam, S., & Croney, C. (2022) · Animals · doi:10.3390/ani12202820 · Cited by 6
Automatic measurements of fetal intracranial volume from 3D ultrasound scans
Caspi, Y., de Zwarte, S. M. C., Iemenschot, I. J., Lumbreras, R., de Heus, R., Bekker, M. N., & Pol, H. E. H. (2022) · Frontiers in Neuroimaging · doi:10.3389/fnimg.2022.996702 · Cited by 5
Chotard, H., Bard, K. A., Micheletta, J., & Davila‐Ross, M. (2022) · American Journal of Primatology · doi:10.1002/ajp.23451 · Cited by 6
Chang, C., Lu, H.-Y., Chang, Y., Tsai, C., & Yen, C. (2022) · BMC Public Health · doi:10.1186/s12889-022-14604-9 · Cited by 6
Psychosocial Correlates of Recreational Screen Time among Adolescents
Roussel-Ouellet, J., Beaulieu, D., Vézina‐Im, L., Turcotte, S., Labbé, V., & Bouchard, D. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192416719 · Cited by 6
Timmerman, I., Macaire, C., Hanne‐Poujade, S., et al. (2022) · Sensors · doi:10.3390/s22239533 · Cited by 6
Fiabilidad y validez de las pruebas v-sit-and-reach y toe-touch en preescolares
Pérez-Vigo, C., Sánchez-Lastra, M. A., Martínez‐de‐Quel, Ó., et al. (2022) · Revista Internacional de Medicina y Ciencias de la Actividad Física y del Deporte · doi:10.15366/rimcafd2022.88.015 · Cited by 6
Tell Me What To Do Not How To Do It: Influence of Creative Outcome and Process Goals on Creativity
Keith, M. G., & Jagacinski, C. M. (2023) · The Journal of Creative Behavior · doi:10.1002/jocb.577 · Cited by 6
Ross, D. E., Seabaugh, J. D., Seabaugh, J. M., Alvarez, C., Ellis, L. P., Powell, C., Reese, C., Cooper, L., Shepherd, K. M., & Initiative, F. T. A. D. N. (2022) · Concussion · doi:10.2217/cnc-2022-0003 · Cited by 6
Marginal and Canonical Babbling in 10 Infants at Risk for Cerebral Palsy
Long, H. L., & Hustad, K. C. (2023) · American Journal of Speech-Language Pathology · doi:10.1044/2022_ajslp-22-00165 · Cited by 6
Williams, M. N., & Apicella, C. L. (2023) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2023.02.002 · Cited by 6
Clinical profiles and intraoperative identification of complex glands in stage I lung adenocarcinoma
Ding, Q., Chen, D., Shen, S., et al. (2023) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezad073 · Cited by 6
Martínez-Cal, J., Molina‐Torres, G., Carrasco-Vega, E., Barni, L., Ventura‐Miranda, M. I., & González‐Sánchez, M. (2023) · Healthcare · doi:10.3390/healthcare11060912 · Cited by 6
Hocaoğlu, M., Murtagh, F. E., Walshe, C., Chambers, R. L., Maddocks, M. M., Sleeman, K. E., Oluyase, A. O., Dunleavy, L., Bradshaw, A., Bajwah, S., et al. (2023) · Health and Quality of Life Outcomes · doi:10.1186/s12955-023-02102-4 · Cited by 7
Varón, J. A. G., Brown, J. L., & Downer, J. T. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1065749 · Cited by 7
Latent profile analysis of students’ perception of German classroom climate: outcomes and covariates
Zhou, J. T., & Hawrot, A. (2023) · Learning Environments Research · doi:10.1007/s10984-023-09471-z · Cited by 6
Razzouk, J. A., Ramos, O., Mehta, S., et al. (2023) · Operative Neurosurgery · doi:10.1227/ons.0000000000000696 · Cited by 6
Comparing Common Retinal Vessel Caliber Measurement Software with an Automatic Deep Learning System
He, S., Bulloch, G., Zhang, L., et al. (2023) · Current Eye Research · doi:10.1080/02713683.2023.2212881 · Cited by 6
Validation of the revised scale of students’ attitudes towards research
Akour, M. M., Damra, J. K. M., Ali, T. M. A., Ghaith, S. M., Ghbari, T. A., & Shammout, N. A. (2023) · Studies in Higher Education · doi:10.1080/03075079.2023.2220700 · Cited by 7
Kibar, P. N. (2023) · Journal of Visual Literacy · doi:10.1080/1051144x.2023.2222260 · Cited by 6
Hofer, J., Kerpen, E., Busch, H., Lehmann, M., & Menon, J. A. (2023) · Personality and Social Psychology Bulletin · doi:10.1177/01461672231181938 · Cited by 6
Better Understanding Oral Communication Skills: The Impact of Perceived Personality Traits
Martín‐Raugh, M. P., Roohr, K. C., Leong, C. W., et al. (2023) · American Journal of Distance Education · doi:10.1080/08923647.2023.2235950 · Cited by 6
Investigating cycle shifts in women's clothing style and grooming
Stern, J., Ostermann, S., & Penke, L. (2023) · British Journal of Social Psychology · doi:10.1111/bjso.12681 · Cited by 6
Developing a behaviour rubric for the practical model of ethical behaviour for clinical nursing
辻尾有利子, Yasuda, M., Hattori, M., Yoshioka, T., Nakamura, N., Nakata, M., Teramukai, S., Minemura, Y., & Azuma, T. (2023) · Nursing Open · doi:10.1002/nop2.1992 · Cited by 6
Díaz‐Fernández, Á., Martínez, A. R. O., Cortés‐Pérez, I., Ibáñez‐Vera, A. J., Obrero‐Gaitán, E., & Lomas‐Vega, R. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12186045 · Cited by 6
A window into your status: Environment-based social class’s effect on virtual leadership
Loignon, A. C., Johnson, M. A., & Veestraeten, M. (2023) · The Leadership Quarterly · doi:10.1016/j.leaqua.2023.101735 · Cited by 8
Meyer, J. N., Pan, W., Ryde, I. T., Alexander, T., Klein‐Adams, J. C., Ndirangu, D. S., & Falvo, M. J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287412 · Cited by 6
Igarashi, T., Tani, Y., Takeda, R., & Asakura, T. (2023) · Scientific Reports · doi:10.1038/s41598-023-46725-5 · Cited by 6
Feigin, K., Snyder, C. J., Tai, J., et al. (2023) · Journal of Veterinary Dentistry · doi:10.1177/08987564231221344 · Cited by 6
O’Bryan, S. J., Hiam, D., & Lamon, S. (2024) · European Journal of Applied Physiology · doi:10.1007/s00421-023-05395-x · Cited by 5
Testing perceivers’ accuracy and accuracy awareness when forming personality impressions from faces
Jaeger, B., Sleegers, W. W. A., Stern, J., Penke, L., & Jones, A. L. (2024) · European Journal of Personality · doi:10.1177/08902070231225728 · Cited by 11
Split‐sample reliability estimation in health care quality measurement: Once is not enough
Nieser, K. J., & Harris, A. H. S. (2024) · Health Services Research · doi:10.1111/1475-6773.14310 · Cited by 6
Online review data analytics to explore factors affecting consumers’ airport recommendations
Kim, C. G., Lee, J., & Lee, K. C. (2024) · Information Technology and People · doi:10.1108/itp-07-2022-0542 · Cited by 6
Yeryomenko, N. (2012) · Scholarship at UWindsor (University of Windsor) · Cited by 6
A normative study of the Nine Hole Peg Test in Bangladesh
Lindstrom-Hazel, D., Aeyman, U., Hossain, S. S., et al. (2015) · Work · doi:10.3233/wor-151996 · Cited by 6
Teams and performance appraisal
Valle, M., & Davis, K. (1999) · Team Performance Management · doi:10.1108/13527599910304912 · Cited by 6
Platzer, S., Wildburger, A., Lorenzoni, M., Jakse, N., Riedl, R., Weiglein, A. H., Wegscheider, W., & Kirmeier, R. (2012) · Clinical Implant Dentistry and Related Research · doi:10.1111/j.1708-8208.2012.00472.x · Cited by 6
Røe, Y., Haldorsen, B., Svege, I. C., & Bergland, A. (2013) · Physiotherapy Research International · doi:10.1002/pri.1555 · Cited by 5
Thunborg, C., Wågert, P. V. H., Ivarsson, A. B., & Söderlund, A. (2015) · Physical & Occupational Therapy In Geriatrics · doi:10.3109/02703181.2015.1065939 · Cited by 7
Rocha-Júnior, V. A., Bottaro, M., Pereira, M. C., Ferreira‐Júnior, J. B., do Carmo, J. C., Brown, L. E., & Nascimento, F. A. D. O. (2015) · Isokinetics and Exercise Science · doi:10.3233/ies-140558 · Cited by 6
Validity and reliability of a new test for Turkish-speaking aphasic patients: Ege Aphasia Test
Çalış, F. A., On, A. Y., & Durmaz, B. (2013) · Neurorehabilitation · doi:10.3233/nre-130833 · Cited by 6
Manca, A., Solinas, G., Dragone, D., Dvir, Z., & Deriu, F. (2015) · Journal of Electromyography and Kinesiology · doi:10.1016/j.jelekin.2015.06.002 · Cited by 6
Tashiro, Y., Okazaki, K., & Iwamoto, Y. (2015) · Open Access Journal of Sports Medicine · doi:10.2147/oajsm.s80809 · Cited by 6
Arensman, R. M., Pisters, M. F., Ginkel, J. M. D. M., et al. (2016) · Physical Therapy · doi:10.2522/ptj.20150265 · Cited by 6
A study of long‐term f MRI reproducibility using data‐driven analysis methods
Song, X., Panych, L. P., Chou, Y., & Chen, N. (2014) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.22111 · Cited by 6
Lafave, M. R., & Katz, L. (2014) · Journal of Athletic Training · doi:10.4085/1062-6050-49.1.12 · Cited by 6
Evaluating the Utility, Reliability, and Validity of a Resident Performance Evaluation Instrument
Tomisato, S., Venter, J., Weller, J. M., et al. (2014) · Academic Psychiatry · doi:10.1007/s40596-014-0134-7 · Cited by 6
Center of pressure velocity autocorrelation as a new measure of postural control during quiet stance
Hernandez, M. E., Stevenson, C., Snider, J., et al. (2013) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2013.6696172 · Cited by 6
Vogel, A. P., Block, S., Kefalianos, E., Onslow, M., Eadie, P. A., Barth, B., Conway, L. J., Mundt, J. C., & Reilly, S. (2014) · International Journal of Speech-Language Pathology · doi:10.3109/17549507.2014.923511 · Cited by 6
Reliability of self-reports by therapists using the Therapist Action Scale.
Xenakis, S. N., Hoyt, M. F., Marmar, C. R., & Horowitz, M. J. (1983) · Psychotherapy · doi:10.1037/h0090212 · Cited by 6
Short-Term Variability of Heart Rate Turbulence in Chronic Heart Failure
Manzano‐Fernández, S., Pastor‐Pérez, F. J., Barquero‐Pérez, Ó., Pascual-Figal, D. A., Goya–Esteban, R., Rojo‐Álvarez, J. L., Caamaño, A. J., Martinez-Espejo, M. D. M., Januzzi, J. L., Valdés, M., et al. (2011) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2011.05.007 · Cited by 6
Verma, S. K., Mitchell, D. G., Lakhman, Y., et al. (2007) · Abdominal Imaging · doi:10.1007/s00261-007-9333-y · Cited by 6
Using Response Latency Measures for a Biographical Inventory
Stricker, L. J., & Alderton, D. L. (1999) · Military Psychology · doi:10.1207/s15327876mp1102_3 · Cited by 6
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
