The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 4,601–4,700 of 23,883
Melchers, K. G., LIENHARDT, N., AARBURG, M. V., & Kleinmann, M. (2011) · Personnel Psychology · doi:10.1111/j.1744-6570.2010.01202.x · Cited by 63
Lee, P. C., & Moss, C. J. (2012) · Journal of comparative psychology · doi:10.1037/a0026566 · Cited by 63
Aguinis, H., Mazurkiewicz, M. D., & Heggestad, E. D. (2009) · Personnel Psychology · doi:10.1111/j.1744-6570.2009.01144.x · Cited by 63
Yu, K., Lin, W., Wang, L., Ma, J., Wei, W., Wang, H., Guo, W., & Shi, J. (2014) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12101 · Cited by 70
Faceness or complexity: Evidence from newborn visual tracking of facelike stimuli
Easterbrook, M. A., Kisilevsky, B. S., Hains, S. M. J., & Muir, D. W. (1999) · Infant Behavior and Development · doi:10.1016/s0163-6383(99)80003-4 · Cited by 66
Todd, G., Taylor, J. L., Butler, J. E., Martin, P. G., Gorman, R. B., & Gandevia, S. C. (2007) · Journal of Applied Physiology · doi:10.1152/japplphysiol.00962.2006 · Cited by 62
Davies, P. L., Soon, P. L., Young, M., et al. (2004) · Physical & Occupational Therapy In Pediatrics · doi:10.1300/j006v24n03_03 · Cited by 63
Comparison of automatic summarisation methods for clinical free text notes
Moen, H., Peltonen, L., Heimonen, J., Airola, A., Pahikkala, T., Salakoski, T., & Salanterä, S. (2016) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2016.01.003 · Cited by 65
Protecting the turf: The effect of territorial marking on others’ creativity.
Brown, G., & Baer, M. (2015) · Journal of Applied Psychology · doi:10.1037/a0039254 · Cited by 65
Antony, B., Driban, J. B., Price, L. L., et al. (2016) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2016.08.004 · Cited by 63
Study of chromatic adaptation using memory color matches, Part I: neutral illuminants
Smet, K. A. G., Zhai, Q., Luo, M. R., et al. (2017) · Optics Express · doi:10.1364/oe.25.007732 · Cited by 63
Girkin, C. A., Fazio, M. A., Yang, H., et al. (2017) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.17-21842 · Cited by 63
Ista, E., Beest, H. T., van Rosmalen, J., de Hoog, M., Tibboel, D., van Beusekom, B. S., & van Dijk, M. (2017) · Australian Critical Care · doi:10.1016/j.aucc.2017.07.006 · Cited by 68
Versteeg, A. L., Sahgal, A., Rhines, L. D., Sciubba, D. M., Schuster, J. M., Weber, M. H., Varga, P. P., Boriani, S., Bettegowda, C., Fehlings, M. G., et al. (2018) · Cancer · doi:10.1002/cncr.31240 · Cited by 63
Iannetta, D., Inglis, E. C., Soares, R. N., et al. (2018) · Microvascular Research · doi:10.1016/j.mvr.2018.10.001 · Cited by 63
Fraval, S., Hammond, J., Wichern, J., et al. (2018) · Experimental Agriculture · doi:10.1017/s0014479718000388 · Cited by 63
Individual differences in object recognition.
Richler, J. J., Tomarken, A. J., Sunday, M. A., Vickery, T. J., Ryan, K. F., Floyd, R. J., Sheinberg, D. L., Wong, A. C. ‐., & Gauthier, L. (2019) · Psychological Review · doi:10.1037/rev0000129 · Cited by 68
Ötting, S. K., Masjutin, L., Steil, J. J., & Maier, G. W. (2020) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820966433 · Cited by 74
Huguet, G., Schramm, C., Douard, E. A., Tamer, P., Main, A., Monin, P., England, J., Jizi, K., Renne, T., Poirier, M., et al. (2021) · Molecular Psychiatry · doi:10.1038/s41380-020-00985-z · Cited by 64
Predicting recurrence of major depressive disorder in young adults: A prospective study.
Hart, A. B., Craighead, W. E., & Craighead, L. W. (2001) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.110.4.633 · Cited by 64
Fritz, J. M., & Piva, S. R. (2003) · Spine · doi:10.1097/01.brs.0000067270.50897.db · Cited by 63
30-Month Stability of Personality Disorder Diagnoses in Depressed Outpatients
Ferro, T., Klein, D. N., Schwartz, J. E., Kasch, K. L., & Leader, J. B. (1998) · American Journal of Psychiatry · doi:10.1176/ajp.155.5.653 · Cited by 63
Parker, M. G., Bennett, M. J., Hieb, M. A., et al. (2003) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2003.33.12.719 · Cited by 63
Effect of ankle dorsiflexion range of motion on rearfoot motion during walking
Cornwall, M., & McPoil, T. G. (1999) · Journal of the American Podiatric Medical Association · doi:10.7547/87507315-89-6-272 · Cited by 63
Exercise-Related Leg Pain in Collegiate Cross-Country Athletes: Extrinsic and Intrinsic Risk Factors
Reinking, M. F., Austin, T. M., & Hayes, A. M. (2007) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2007.2534 · Cited by 63
LeResche, L., Burgess, J., & Dworkin, S. F. (1988) · Journal of Dental Research · doi:10.1177/00220345880670010601 · Cited by 63
Reliability testing of the Six Area, Six Sign Atopic Dermatitis severity score
Charman, C. R., Venn, A. J., & Williams, H. C. (2002) · British Journal of Dermatology · doi:10.1046/j.1365-2133.2002.04644.x · Cited by 63
Training Users in the Gross Motor Function Measure: Methodological and Practical Issues
Russell, D. J., Rosenbaum, P., Lane, M. E., Gowland, C. A., Goldsmith, C. H., Boyce, W. F., & Plews, N. (1994) · Physical Therapy · doi:10.1093/ptj/74.7.630 · Cited by 62
Feczko, E., Augustinack, J. C., Fischl, B. R., & Dickerson, B. C. (2007) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2007.07.023 · Cited by 63
Social Functioning of Patients With Schizophrenia in High-Income Welfare Societies
Melle, I. S., Friis, S., Hauff, E., et al. (2000) · Psychiatric Services · doi:10.1176/appi.ps.51.2.223 · Cited by 63
Predictors of pain self-report in nursing home residents
Weiner, D. K., Peterson, B. L., Logue, P. E., & Keefe, F. J. (1998) · Aging Clinical and Experimental Research · doi:10.1007/bf03339888 · Cited by 63
Westerman, M. A. (1990) · Developmental Psychology · doi:10.1037/0012-1649.26.4.621 · Cited by 63
Suboptimal phenotypic reliability impedes reproducible human neurosciencePreprint
Nikolaidis, A., Chen, A. A., He, X., Shinohara, R. T., Vogelstein, J. T., Milham, M. P., & Shou, H. (2022) · Research Square · doi:10.1101/2022.07.22.501193 · Cited by 60
The impact of sleep, stress, and depression on postpartum weight retention: A systematic review
Xiao, R. S., Kroll-Desrosiers, A. R., Goldberg, R. J., Pagoto, S. L., Person, S. D., & Waring, M. E. (2014) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2014.09.016 · Cited by 62
Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., Okamura, K., Daleiden, E. L., Bearman, S. K., Weisz, J. R., & The Research Network on Youth Mental Health (2013) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2012.700505 · Cited by 62
Schindler, S., Schönknecht, P., Schmidt, L., Anwander, A., Strauß, M., Trampel, R., Bazin, P., Möller, H. E., Hegerl, U., Turner, R., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066394 · Cited by 63
Savory, S., Agim, N. G., Mao, R., Peter, S., Wang, C., Maldonado, G., Dietert, J. B., Lieu, T. J., Wang, C., Pretzlaff, K., et al. (2013) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2013.09.017 · Cited by 62
Elorza, U., Aritzeta, A., & Ayestarán, S. (2011) · The International Journal of Human Resource Management · doi:10.1080/09585192.2011.561956 · Cited by 66
Concurrent Validity of the Zeno Walkway for Measuring Spatiotemporal Gait Parameters in Older Adults
Vallabhajosula, S., Humphrey, S., Cook, A., & Freund, J. E. (2017) · Journal of Geriatric Physical Therapy · doi:10.1519/jpt.0000000000000168 · Cited by 64
Linking trauma to posttraumatic growth and mental health through emotional and cognitive creativity.
Orkibi, H., & Ram-Vlasov, N. (2018) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/aca0000193 · Cited by 78
Haller, S. P., Kircanski, K., Stringaris, A. K., Clayton, M., Bui, H. N. T., Agorsor, C. F., Cárdenas, S. I., Towbin, K. E., Pine, D. S., Leibenluft, E., et al. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.10.005 · Cited by 62
Gaining clarity about emotion differentiation
Thompson, R. J., Springstein, T., & Boden, M. T. (2021) · Social and Personality Psychology Compass · doi:10.1111/spc3.12584 · Cited by 80
Aboyans, V., Lacroix, P., Doucet, S., Preux, P., Criqui, M. H., & Laskar, M. (2008) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2008.01784.x · Cited by 61
Cornélis, F., Tricaud, E., Lasserre, A. S., et al. (2014) · European Radiology · doi:10.1007/s00330-014-3380-x · Cited by 62
Inter- and intra-rater reliability of the GAITRite system among individuals with sub-acute stroke
Wong, J. S., Jasani, H., Poon, V., et al. (2014) · Gait & Posture · doi:10.1016/j.gaitpost.2014.02.007 · Cited by 62
Prudlo, J., Bißbort, C., Glass, Ä., Großmann, A., Hauenstein, K., Benecke, R., & Teipel, S. J. (2012) · Journal of Neurology · doi:10.1007/s00415-012-6420-y · Cited by 62
T2* measurement of the knee articular cartilage in osteoarthritis at 3T
Newbould, R. D., Miller, S. R., Toms, L., Swann, P., Tielbeek, J. A. W., Gold, G. E., Strachan, R. K., Taylor, P. C., Matthews, P. M., & Brown, A. P. (2012) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.23598 · Cited by 62
Auditory‐Perceptual Scaling and Quality of Life in Tracheoesophageal Speakers
Eadie, T. L., & Doyle, P. C. (2004) · The Laryngoscope · doi:10.1097/00005537-200404000-00030 · Cited by 62
Karpova, A., Ranganathan, A., Davis, A. M., et al. (2012) · Spine · doi:10.1097/brs.0b013e3182672307 · Cited by 62
Morsbach, F., Wurnig, M. C., Kunz, D. M., et al. (2013) · European Radiology · doi:10.1007/s00330-013-2885-z · Cited by 62
O’Leary, S. A., Doyle, B. J., & McGloughlin, T. M. (2013) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2013.05.003 · Cited by 62
Bohannon, R. W. (1995) · Perceptual and Motor Skills · doi:10.2466/pms.1995.80.2.379 · Cited by 62
Ip, W. Y., & Martin, C. R. (2006) · Psychology Health & Medicine · doi:10.1080/13548500500155750 · Cited by 62
Paris, S., Bitter, K., Renz, H., et al. (2009) · Microscopy Research and Technique · doi:10.1002/jemt.20701 · Cited by 62
The sources of leader violence: A comparison of ideological and non-ideological leaders
Mumford, M. D., Espejo, J., Hunter, S. T., et al. (2007) · The Leadership Quarterly · doi:10.1016/j.leaqua.2007.03.005 · Cited by 62
The Discussion Coding System (DCS)—A New Instrument for Analyzing Communication Processes
Schermuly, C. C., & Schöll, W. (2012) · Communication Methods and Measures · doi:10.1080/19312458.2011.651346 · Cited by 62
Psychometric properties of four fear of falling rating scales in people with Parkinson’s disease
Jonasson, S. B., Nilsson, M. H., & Lexell, J. E. (2014) · BMC Geriatrics · doi:10.1186/1471-2318-14-66 · Cited by 66
Test–retest reliability of spatial and temporal gait parameters of people with Alzheimer's disease
Wittwer, J. E., Webster, K. E., Andrews, P. T., et al. (2008) · Gait & Posture · doi:10.1016/j.gaitpost.2008.01.007 · Cited by 62
An interrater reliability study of the Braden scale in two nursing homes
Kottner, J., & Dassen, T. W. N. (2008) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2008.02.007 · Cited by 61
Spinal Cord Injury Functional Ambulation Profile
Musselman, K. E., Brunton, K., Lam, T., et al. (2011) · Neurorehabilitation and neural repair · doi:10.1177/1545968310381250 · Cited by 62
Dietary assessment in the California Teachers Study: reproducibility and validity
Horn‐Ross, P. L., Lee, V. S., Collins, C. N., STEWART, S. L., Canchola, A. J., Lee, M. M., Reynolds, P., Clarke, C. A., Bernstein, L., & Stram, D. O. (2008) · Cancer Causes & Control · doi:10.1007/s10552-008-9124-y · Cited by 63
Manns, P. J., & Haennel, R. G. (2012) · Stroke Research and Treatment · doi:10.1155/2012/247165 · Cited by 60
The structure of temperament in preschoolers: A two-stage factor analytic approach.
Dyson, M. W., Olino, T. M., Durbin, C. E., Goldsmith, H. H., & Klein, D. N. (2011) · Emotion · doi:10.1037/a0025023 · Cited by 69
Domingo, A., & Lam, T. (2014) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-11-167 · Cited by 62
Kyvelidou, A., Harbourne, R. T., Shostrom, V., & Stergiou, N. (2010) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2010.06.027 · Cited by 62
Simmons, R., Chambless, D. L., & Gordon, P. C. (2008) · Family Process · doi:10.1111/j.1545-5300.2008.00261.x · Cited by 67
Within‐subject variation of seminal parameters in men with infertile marriages
Francavilla, F., Barbonetti, A., Necozione, S., Santucci, R., Cordeschi, G., Macerola, B., & Francavilla, S. (2007) · International Journal of Andrology · doi:10.1111/j.1365-2605.2006.00727.x · Cited by 62
Vaskinn, A., Sundet, K. S., Friis, S., Simonsen, C., Birkenaes, A. B., Jónsdóttir, H., Ringen, P. A., & Andreassen, O. A. (2008) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617708080314 · Cited by 61
Walker, A., Recker, M., Robertshaw, M., Osen, J., Leary, H., Ye, L., & Sellers, L. (2011) · Interdisciplinary Journal of Problem-based Learning · doi:10.7771/1541-5015.1255 · Cited by 76
Balestrieri, M., Isola, M., Bonn, R., Tam, T., Vio, A., Linden, M., & Maso, E. (2012) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796012000480 · Cited by 62
Webb, A. L., Carding, P. N., Deary, I. J., MacKenzie, K., Steen, I. N., & Wilson, J. A. (2007) · The Journal of Laryngology & Otology · doi:10.1017/s0022215107007177 · Cited by 61
The Development of a Standardized Skill Assessment for Junior Volleyball Players
Gabbett, T. J., & Georgieff, B. (2006) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.1.2.95 · Cited by 62
van Horn, M. L. (2003) · Educational and Psychological Measurement · doi:10.1177/0013164403251317 · Cited by 62
In-Home Occupational Performance Evaluation (I–HOPE)
Stark, S. L., Somerville, E. K., & Morris, J. C. (2010) · American Journal of Occupational Therapy · doi:10.5014/ajot.2010.08065 · Cited by 63
Coolidge, T., Hillstead, M. B., Farjo, N., Weinstein, P., & Coldwell, S. E. (2010) · BMC Oral Health · doi:10.1186/1472-6831-10-12 · Cited by 62
Aiyer, A. A., Dall, G. F., Shub, J., et al. (2015) · Foot & Ankle International · doi:10.1177/1071100715620894 · Cited by 62
Lane, S. P., Steinley, D., & Sher, K. J. (2016) · Psychological Medicine · doi:10.1017/s0033291716000404 · Cited by 61
Parra‐Herran, C., Taljaard, M., Djordjevic, B., et al. (2016) · Modern Pathology · doi:10.1038/modpathol.2016.86 · Cited by 62
Lee, Y. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.05.004 · Cited by 63
Christidi, F., Karavasilis, E., Samiotis, K., Bisdas, S., & Papanikolaou, N. K. (2016) · European Journal of Radiology Open · doi:10.1016/j.ejro.2016.06.002 · Cited by 64
Comparison of two- and three-dimensional measurement of the Cobb angle in scoliosis
Lechner, R., Putzer, D., Dammerer, D., et al. (2016) · International Orthopaedics · doi:10.1007/s00264-016-3359-0 · Cited by 62
Technology-Enhanced Simulation for Health Professions Education
Cook, D. A., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hamstra, S. J. (2012) · Cited by 62
van der Werf, A., Dekker, I. M., Meijerink, M. R., Wierdsma, N., de van der Schueren, M. A. E., & Langius, J. A. E. (2017) · Clinical Physiology and Functional Imaging · doi:10.1111/cpf.12422 · Cited by 65
Raper, D., Witchalls, J. B., Philips, E. J., Knight, E., Drew, M. K., & Waddington, G. S. (2017) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2017.06.010 · Cited by 62
Cao, W., Cui, S., Yang, L., et al. (2017) · Antioxidants and Redox Signaling · doi:10.1089/ars.2017.7006 · Cited by 62
Intracranial source activity (eLORETA) related to scalp‐level asymmetry scores and depression status
Smith, E. E., Cavanagh, J. F., & Allen, J. J. B. (2017) · Psychophysiology · doi:10.1111/psyp.13019 · Cited by 68
The role of fMRI in drug development
Carmichael, O. T., Schwarz, A. J., Chatham, C. H., Scott, D., Turner, J. A., Upadhyay, J., Coimbra, A., Goodman, J. A., Baumgartner, R., English, B. A., et al. (2017) · Drug Discovery Today · doi:10.1016/j.drudis.2017.11.012 · Cited by 60
Gledhill, A., Forsdyke, D., & Murray, E. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-097694 · Cited by 62
A comparison of the electrocortical response to monetary and social reward
Distefano, A., Jackson, F. L., Levinson, A. R., Infantolino, Z. P., Jarcho, J. M., & Nelson, B. D. (2018) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsy006 · Cited by 67
Daloia, L. M. T., Leonardi-Figueiredo, M. M., Martínez, E. Z., & Mattiello‐Sverzut, A. C. (2018) · Brazilian Journal of Physical Therapy · doi:10.1016/j.bjpt.2018.04.006 · Cited by 64
Park, Y. S., Shin, W. C., Lee, S. M., et al. (2018) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-018-0767-4 · Cited by 62
Park, J., Chae, E. Y., Cha, J. H., et al. (2018) · European Journal of Radiology · doi:10.1016/j.ejrad.2018.09.032 · Cited by 62
Hippocampal CA1 subfield predicts episodic memory impairment in Parkinson's disease
La, C., Linortner, P., Bernstein, J. D., Cruadhlaoich, M. A. I. U., Fenesy, M. C., Deutsch, G. K., Rutt, B. K., Tian, L., Wagner, A. D., Zeineh, M., et al. (2019) · NeuroImage Clinical · doi:10.1016/j.nicl.2019.101824 · Cited by 62
Wilhelm, S., Weingarden, H. M., Greenberg, J. L., McCoy, T. H., Ladis, I., Summers, B. J., Matic, A., & Harrison, O. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.03.007 · Cited by 64
Sánchez, L. E., Rundo, L., Gill, A. B., Hoare, M., Serrão, E. M., & Sala, E. (2021) · Scientific Reports · doi:10.1038/s41598-021-87598-w · Cited by 62
Reliability of the hip examination in osteoarthritis: Effect of standardization
Cibere, J., Thorne, A., Bellamy, N., Greidanus, N. V., Chalmers, A., Mahomed, N. N., Shojania, K., Kopec, J. A., & Esdaile, J. M. (2008) · Arthritis Care & Research · doi:10.1002/art.23310 · Cited by 62
Carabello, R. J., Reid, K. F., Clark, D. J., Phillips, E. M., & Fielding, R. A. (2010) · Aging Clinical and Experimental Research · doi:10.1007/bf03337729 · Cited by 63
The importance of ward atmosphere in inpatient treatment of schizophrenia on short-term units
Melle, I. S., Friis, S., Hauff, E., et al. (1996) · Psychiatric Services · doi:10.1176/ps.47.7.721 · Cited by 62
The stereotyped behavior scale: psychometric properties and norms
Rojahn, J., Matlock, S. T., & Tassé, M. J. (2000) · Research in Developmental Disabilities · doi:10.1016/s0891-4222(00)00057-3 · Cited by 62
Classroom teaching behaviors related to college teaching effectiveness
Murray, H. G. (1985) · New Directions for Teaching and Learning · doi:10.1002/tl.37219852305 · Cited by 62
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
