The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,342
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 13,701–13,800 of 23,881
Oh, J., Kuenze, C. M., Signorile, J. F., Andersen, M. S., Letter, M. I., Best, T. M., Ripic, Z., Emerson, C. P., & Eltoukhy, M. A. (2020) · Gait & Posture · doi:10.1016/j.gaitpost.2020.12.025 · Cited by 10
Smith, R. D., McHugh, G. A., Quicke, J. G., Dziedzic, K., & Healey, E. L. (2021) · Musculoskeletal Care · doi:10.1002/msc.1540 · Cited by 13
Colombo, A., Saia, G., Azzena, A. A., Rossi, A., Zugni, F., Pricolo, P., Summers, P. E., Marvaso, G., Grimm, R., Bellomi, M., et al. (2021) · Diagnostics · doi:10.3390/diagnostics11030499 · Cited by 11
Youssef, A., Brunelli, E., Fiorentini, M., et al. (2021) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.23649 · Cited by 11
Explicit Facial Expression Transfer via Fine-Grained Representations
(9458226), Z. S., (13455051), H. Z., (13455054), J. T., Lu, X., & (13323042), L. M. (2021) · Figshare · Cited by 11
Nocera, V. G., Wood, A. P., Wozencroft, A. J., & Coe, D. P. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18105367 · Cited by 11
Zhou, Z., Yang, J., Wang, S., et al. (2021) · Medicine · doi:10.1097/md.0000000000026212 · Cited by 11
Kruisbrink, M., Crutzen, R., Kempen, G. I. J. M., et al. (2021) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2021.104469 · Cited by 11
Chu, Y., Ma, G., Xu, X., et al. (2021) · Acta Radiologica · doi:10.1177/02841851211029080 · Cited by 11
Weiß, A., Yokoyama, C., Hayashi, T., & Inoue‐Murayama, M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0238663 · Cited by 13
Donohue, C., Khalifa, Y., Perera, S., Sejdić, E., & Coyle, J. L. (2021) · Dysphagia · doi:10.1007/s00455-021-10368-3 · Cited by 11
Zimmerer, A., Löchel, J., Schoon, J., Janz, V., & Wassilew, G. I. (2021) · Scientific Reports · doi:10.1038/s41598-021-99011-7 · Cited by 11
Ramis-Alario, A., Tarazona‐Álvarez, B., Diago, M. P., et al. (2021) · Medicina oral, patología oral y cirugía bucal · doi:10.4317/medoral.24447 · Cited by 11
Suh, J. W., Jang, H.-S., & Park, H. (2021) · Foot and Ankle Surgery · doi:10.1016/j.fas.2021.11.005 · Cited by 12
Choi, T. M., Lijten, O. W., Mathijssen, I. M. J., et al. (2021) · Clinical Oral Investigations · doi:10.1007/s00784-021-04275-y · Cited by 11
Gater, A., Nelsen, L. M., Coon, C. D., Eremenco, S., O’Quinn, S., Khan, A. H., Eckert, L., Staunton, H., Bonner, N., Hall, R., et al. (2021) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2021.11.026 · Cited by 12
Woodard, G. S., Triplett, N. S., Frank, H. E., Harrison, J. P., Robinson, S., & Dorsey, S. (2021) · Implementation Research and Practice · doi:10.1177/26334895211057883 · Cited by 13
Hill, C. E., Heales, L., Stanton, R., & Kean, C. O. (2022) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2022.2030445 · Cited by 11
Schwartz, K., Theis, E. N., Bunting, K., et al. (2022) · Microvascular Research · doi:10.1016/j.mvr.2022.104363 · Cited by 11
The French MBAA2 sentence recognition in noise test for cochlear implant users
James, C. J., Laborde, M., Algans, C., et al. (2022) · International Journal of Audiology · doi:10.1080/14992027.2022.2045368 · Cited by 11
Defining and Assessing Visibility Enablers in the Industrial Construction Supply Chain
Dharmapalan, V., O’Brien, W. J., & Morrice, D. J. (2022) · Journal of Management in Engineering · doi:10.1061/(asce)me.1943-5479.0001047 · Cited by 11
Wang, A. Y.-D., Saini, H., Tingen, J. N., et al. (2022) · World Neurosurgery · doi:10.1016/j.wneu.2022.04.008 · Cited by 11
Test-retest reliability of multi-parametric maps (MPM) of brain microstructure
Aye, N., Lehmann, N., Kaufmann, J., Heinze, H., Düzel, E., Taubert, M., & Ziegler, G. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119249 · Cited by 11
Zhu, B., Liu, Y., Wang, H., Duan, F., Mi, L., & Liang, Y. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-057736 · Cited by 11
Guldberg–Møller, J., Mogensen, M., Ellegaard, K., et al. (2022) · RMD Open · doi:10.1136/rmdopen-2021-002109 · Cited by 11
Li, X., Liu, Q., Hu, B., et al. (2021) · Journal of Cancer Research and Therapeutics · doi:10.4103/jcrt.jcrt_1755_21 · Cited by 11
Larsson, S. L., Brogren, E., Dahlin, L. B., Björkman, A., & Ekstrand, E. (2022) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-022-05511-6 · Cited by 11
Validating the Italian version of the Adult Picky Eating Questionnaire
Menghi, L., Endrizzi, I., Cliceri, D., Zampini, M., Giacalone, D., & Gasperi, F. (2022) · Food Quality and Preference · doi:10.1016/j.foodqual.2022.104647 · Cited by 11
Ramklint, M., Söderberg, P., Tungström, S., Nordenskjöld, A., & Hermansson, L. (2022) · Nordic Journal of Psychiatry · doi:10.1080/08039488.2022.2097738 · Cited by 11
Arthroscopy-assisted bone grafting for the treatment of SNAC stage I without radial styloidectomy
Lee, Y.-K., & Jung, Y.-R. (2022) · Medicine · doi:10.1097/md.0000000000029930 · Cited by 11
Automated interpretation of stress echocardiography reports using natural language processing
Zheng, C., Sun, B., Wu, Y., et al. (2022) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztac047 · Cited by 11
Chokshi, S. K., Tologonova, G., Calixte, R., Yadav, V., Razvi, N. A., Lazar, J., & Kachnowski, S. (2022) · JMIR Formative Research · doi:10.2196/41241 · Cited by 10
Christiaanse, E., Wyss, P. O., Scheel‐Sailer, A., et al. (2022) · NMR in Biomedicine · doi:10.1002/nbm.4856 · Cited by 11
Zhang, C., Greve, C., Verkerke, G. J., et al. (2022) · Sensors · doi:10.3390/s22218342 · Cited by 11
CT radiomics to predict Deauville score 4 positive and negative Hodgkin lymphoma manifestations
Jensen, L. J., Rogasch, J. M. M., Kim, D., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-24227-0 · Cited by 11
Lobet, S., Lambert, C., Foubert, A., Chantrain, V., Roussel, N. A., Meeus, M., Devos, A., Maes, P., Hermans, C. R. J. R., & Penta, M. (2022) · Haemophilia · doi:10.1111/hae.14705 · Cited by 10
de Rousiers, A., Rougereau, G., Rousselin, B., et al. (2023) · Foot and Ankle Surgery · doi:10.1016/j.fas.2023.01.013 · Cited by 11
Validation of the Korean Childhood Autism Rating Scale-2
Lee, S., Yoon, S., & Shin, M. (2023) · Research in Autism · doi:10.1016/j.rasd.2023.102128 · Cited by 11
Sacco, G., Ben-Sadoun, G., Gautier, J., Simon, R., Goupil, M., Laureau, P., Terrien, J., & Annweiler, C. (2023) · BMC Geriatrics · doi:10.1186/s12877-023-03811-7 · Cited by 13
Gui, A., Perelli, D., Rizzo, G., Ferruzza, E., & Mercuriali, E. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1175675 · Cited by 13
Sportel, B. E., Aardema, H., Boonstra, N., Arends, J., Rudd, B., Metz, M. J., Castelein, S., & Pijnenborg, G. H. M. (2023) · BMC Psychiatry · doi:10.1186/s12888-023-04763-3 · Cited by 11
Examining textism convergence in mediated interactions
Adams, A., & Miles, J. (2023) · Language Sciences · doi:10.1016/j.langsci.2023.101568 · Cited by 11
Myong, Y., Park, S.-W., Cho, M., Cho, S. Y., Lee, W. H., Oh, B., & Kim, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-39062-0 · Cited by 11
Toolkits for innovation: how digital technologies empower users in new product development
Schäper, T., Foege, J. N., & Nüesch, S. (2023) · R&D Management · doi:10.1111/radm.12642 · Cited by 11
Çelik, H., Yildirim, A., Ünver, E., et al. (2023) · Measurement in Physical Education and Exercise Science · doi:10.1080/1091367x.2023.2257169 · Cited by 11
Chaniaud, N., Fleury, S., Poussard, B., Olivier Christmann, C., Guitter, T., & Richir, S. (2023) · Design Science · doi:10.1017/dsj.2023.27 · Cited by 11
Three-dimensional architecture of the human subscapularis muscle in vivo
Zhang, Y., Herbert, R., Bilston, L. E., et al. (2023) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2023.111854 · Cited by 11
Singh, P., Hsung, T.-C., Ajmera, D. H., Leung, Y. Y., McGrath, C. P. J., & Gu, M. (2023) · Journal of Dentistry · doi:10.1016/j.jdent.2023.104775 · Cited by 11
Bhimani, R., Ashkani‐Esfahani, S., Mirochnik, K., et al. (2023) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2023.11.025 · Cited by 11
Replication of a neuroimaging biomarker for striatal dysfunction in psychosis
Rubio, J. M., Lencz, T., Cao, H., Kraguljac, N. V., Dhamala, E., Homan, P., Horga, G., Sarpal, D. K., Árgyelán, M., Gallego, J. A., et al. (2024) · Molecular Psychiatry · doi:10.1038/s41380-023-02381-9 · Cited by 12
Wingate, L. A. (2009) · ProQuest LLC eBooks · Cited by 11
Inter‐observer concordance for the recognition of angiotropism in human melanoma
Barnhill, R. L., BUSAM, K. J., From, L., et al. (2011) · Pigment Cell & Melanoma Research · doi:10.1111/j.1755-148x.2011.00854.x · Cited by 11
Hall, D. A., Stebbins, G. T., Litvan, I., et al. (2015) · Movement Disorders Clinical Practice · doi:10.1002/mdc3.12243 · Cited by 11
Homan, M. W. L., Wolkerstorfer, A., Sprangers, M. A. G., et al. (2012) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/j.1468-3083.2012.04568.x · Cited by 11
Wei, X., & Haertel, E. H. (2011) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2010.00196.x · Cited by 11
Sapountzi‐Krepia, D., Raftopoulos, V., Tzavelas, G., et al. (2008) · Midwifery · doi:10.1016/j.midw.2007.12.002 · Cited by 11
Smith, S. R., Chang, J., Schnoebelen, K., Edwards, J. W., Servesko, A. M., & Walker, S. J. (2007) · Journal of Neuropsychology · doi:10.1348/174866407x180800 · Cited by 11
Thirthalli, J., Prasad, M. K., Udupa, K., Kishore⁎, K. R., Sathyaprabha, T., & Gangadhar, B. N. (2009) · Indian Journal of Psychiatry · doi:10.4103/0019-5545.55085 · Cited by 11
Knee Function Assessment in Patients With Meniscus Injury
Naimark, M. B., Kegel, G., O’Donnell, T., et al. (2014) · Orthopaedic Journal of Sports Medicine · doi:10.1177/2325967114550987 · Cited by 11
Direct Magnitude Estimation of Articulation Rate in Boys With Fragile X Syndrome
Zajac, D. J., Harris, A., Roberts, J. E., & Martin, G. E. (2009) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2009/07-0208 · Cited by 12
Standardized standing pelvis-to-floor photographs for the assessment of lower-extremity alignment
Sheehy, L., Cooke, T. D. V., McLean, L., & Culham, E. G. (2014) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2014.12.009 · Cited by 11
Quality appraisal of clinical practice guidelines on glioma
Tian, H., Gou, Y., Pan, Y., Li, Q., Wei, D., Wang, Z., Niu, X., Liang, W., & Zhang, Y. (2014) · Neurosurgical Review · doi:10.1007/s10143-014-0569-z · Cited by 11
Shmueli, A. (2008) · Value in Health · doi:10.1111/j.1524-4733.2008.00435.x · Cited by 11
Contributions of consumer-perceived creativity and beauty to willingness-to-pay for design products
Christensen, B. T., Kristensen, T., & Reber, R. (2015) · International Journal of Design Creativity and Innovation · doi:10.1080/21650349.2014.981216 · Cited by 13
Evaluation of Perpendicular Reflection Intensity for Assessment of Caries Lesion Activity/Inactivity
Neuhaus, K. W., Nyvad, B., Lussi, A., & Jaruszewski, L. (2011) · Caries Research · doi:10.1159/000330530 · Cited by 11
Teixeira, R. P., Monteiro, R., Baptista, R., Barbosa, A., Leite, L., Ribeiro, M. A., Martins, R., Cardim, N. M., & Gonçalves, L. M. M. (2015) · International journal of cardiac imaging · doi:10.1007/s10554-015-0597-y · Cited by 12
Patient Personality and Outcome in Short-Term Psychodynamic Psychotherapy
Cromer, T. D., & Hilsenroth, M. J. (2010) · The Journal of Nervous and Mental Disease · doi:10.1097/nmd.0b013e3181c820fc · Cited by 11
Koo, Y. S., Yang, K., Seok, H. Y., Lee, S. K., Lee, I. K., Cho, Y. W., Lee, J. H., Hwang, H. Y., Kim, K. J., & Jung, K.-Y. (2012) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2012.06.002 · Cited by 11
Riddle, D. L. (2012) · Manual Therapy · doi:10.1016/j.math.2012.05.006 · Cited by 11
Fong, K. W., Dengler, J., Toi, A., Menezes, R. J., Karimzad, Y., & Okun, N. B. (2014) · Journal of Ultrasound in Medicine · doi:10.7863/ultra.33.8.1373 · Cited by 11
Using Mass Distribution Information to Model the Human Thigh for Body Segment Parameter Estimation
Durkin, J. L., Dowling, J. J., & Scholtes, L. (2005) · Journal of Biomechanical Engineering · doi:10.1115/1.1894367 · Cited by 11
Nambi, P. V., Sengupta, R., Krajcer, Z., Muthupillai, R., Strickman, N., & Cheong, B. (2011) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2010.11.027 · Cited by 11
Improving Quantitative CT Perfusion Parameter Measurements Using Principal Component Analysis
Yeung, T. P. C., Dekaban, M., Haan, N. D., Morrison, L. B., Hoffman, L., Bureau, Y., Chen, X., Yartsev, S., Bauman, G. S., & Lee, T.-Y. (2014) · Academic Radiology · doi:10.1016/j.acra.2014.01.015 · Cited by 11
Gorbett, G. E., Morris, S. M., Meacham, B. J., & Wood, C. B. (2014) · Journal of Forensic Sciences · doi:10.1111/1556-4029.12616 · Cited by 11
Glas, B., Stenberg, B., Stenlund, H. C., & Sunesson, A.-L. (2008) · Journal of Environmental Monitoring · doi:10.1039/b808654k · Cited by 11
Yoon, K., Woo, S. T., Ryu, J. A., et al. (2015) · Journal of the Korean Society of Radiology · doi:10.3348/jksr.2015.72.1.57 · Cited by 11
Reimerink, J. J., Marquering, H. A., Vahl, A. C., et al. (2013) · CardioVascular and Interventional Radiology · doi:10.1007/s00270-013-0757-1 · Cited by 11
Automated Analysis of Intima‐Media Thickness
Saba, L., Montisci, R., Famiglietti, L., et al. (2013) · Journal of Ultrasound in Medicine · doi:10.7863/ultra.32.7.1127 · Cited by 11
Tschudi-Madsen, H., Kjeldsberg, M., Natvig, B., Ihlebæk, C., Dalen, I., Straand, J., & Bruusgaard, D. (2013) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2013.01.003 · Cited by 11
Decline in enjoyment of physical education among culturally and linguistically diverse youth
Dudley, D. A., Okely, A. D., Pearson, P., et al. (2013) · International Journal of Quantitative Research in Education · doi:10.1504/ijqre.2013.058308 · Cited by 11
Understanding the science-learning environment: A genetically sensitive approach
Haworth, C. M. A., Davis, O. S. P., Hanscombe, K. B., Kovas, Y. V., Dale, P. S., & Plomin, R. J. (2012) · Learning and Individual Differences · doi:10.1016/j.lindif.2012.07.018 · Cited by 13
Can Ulnar Variance Be Used to Detect Overstuffing After Radial Head Arthroplasty?
Moon, J.-G., Hong, J.-H., Bither, N., et al. (2013) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-013-3277-z · Cited by 11
MacIntyre, N. J., Stavness, C., & Adachi, J. D. (2009) · Clinical Rheumatology · doi:10.1007/s10067-009-1297-6 · Cited by 11
Arnold, B. L., & Perrin, D. H. (1993) · Journal of Sport Rehabilitation · doi:10.1123/jsr.2.4.243 · Cited by 11
A Written Structured Interview by any Other Name is Still a Selection Instrument.
Whetzel, D. L., Baranowski, L., Petro, J. M., Curtin, P. J., & Fisher, J. L. (2003) · Cited by 11
Tully, R., Browne, K., & Craig, L. A. (2015) · Criminal Justice and Behavior · doi:10.1177/0093854814553096 · Cited by 11
Petterwood, J., Dowsey, M. M., Rodda, D., & Choong, P. (2014) · Frontiers in Surgery · doi:10.3389/fsurg.2014.00035 · Cited by 12
Gimmon, Y., Grinshpon, J., Lenoble-Hoskovec, C., Büla, C., & Melzer, I. (2013) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2013.02.001 · Cited by 11
A parentally administered cognitive development assessment for children from 10 to 24 months
Baker, M., Schafer, G., Alcock, K., & Bartlett, S. (2013) · Infant Behavior and Development · doi:10.1016/j.infbeh.2013.01.007 · Cited by 11
Matharu, G. S., Berryman, F., Brash, L., et al. (2015) · International Orthopaedics · doi:10.1007/s00264-014-2644-z · Cited by 11
Schmitz, C. C., Chipman, J. G., Yoshida, K., et al. (2014) · Military Medicine · doi:10.7205/milmed-d-13-00247 · Cited by 11
Khan, M. A., Kuiper, J. H., Sieniawska, C. E., & Richardson, J. B. (2015) · Journal of Orthopaedics · doi:10.1016/j.jor.2015.10.006 · Cited by 11
Looking at Cities in Mexico with Crowds
Santani, D., Ruíz-Correa, S., & Gática-Pérez, D. (2015) · ACM Symposium on Computing for Development (ACM DEV) · doi:10.1145/2830629.2830638 · Cited by 11
Pai, J., Jung, E. J., & Lee, S. (2010) · Seoul Journal of Business · doi:10.35152/snusjb.2010.16.2.006 · Cited by 11
A clinical rating scale for the assessment of facial aging in Indian population
Sen, S., Choudhury, S., Gangopadhyay, A., et al. (2016) · Indian Journal of Dermatology Venereology and Leprology · doi:10.4103/0378-6323.174369 · Cited by 11
Liu, R., Gu, X.-Y., Zhang, J., Yu, L., Chen, W., Wang, K., & Svensson, P. J. (2016) · European Journal Of Oral Sciences · doi:10.1111/eos.12264 · Cited by 11
Reliability analysis of the heart autonomic control parameters during hemodialysis sessions
da Silva, D. M., Macedo, M. C., Lemos, L. B., Vieira, F. C., Pirôpo, U. S., Andrade, H. B., Wessel, N., & Pereira, R. (2016) · Biomedizinische Technik/Biomedical Engineering · doi:10.1515/bmt-2015-0239 · Cited by 11
Frank, M., Peyrard, S., Bobrie, G., & Azizi, M. (2010) · American Journal of Hypertension · doi:10.1038/ajh.2010.47 · Cited by 11
Psychometric properties of fatigue severity and fatigue impact scales in postpolio patients
Öncü, J., Atamaz, F. C., Durmaz, B., et al. (2013) · International Journal of Rehabilitation Research · doi:10.1097/mrr.0b013e3283646b56 · Cited by 11
Bowen, S. W., & Kurz, A. S. (2011) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntr281 · Cited by 11
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
