The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 301–400 of 23,883
MULTIDIMENSIONAL FAMILY THERAPY FOR ADOLESCENT DRUG ABUSE: RESULTS OF A RANDOMIZED CLINICAL TRIAL
Liddle, H. A., Dakof, G. A., Parker, K., Diamond, G. S., Barrett, K., & Tejeda, M. J. (2001) · The American Journal of Drug and Alcohol Abuse · doi:10.1081/ada-100107661 · Cited by 553
Exercise for women receiving adjuvant therapy for breast cancer
Markes, M., Brockow, T., & Resch, K.-L. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005001.pub2 · Cited by 543
Zohar, D. M., & Luria, G. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.2.322 · Cited by 561
RELATIVE STANDING: A FRAMEWORK FOR UNDERSTANDING DEPARTURES OF ACQUIRED EXECUTIVES.
Hambrick, D. C., & Cannella, A. A. (1993) · Academy of Management Journal · doi:10.2307/256757 · Cited by 542
Eliasson, A., Krumlinde‐Sundholm, L., Rösblad, B., Beckung, E., Arner, M., Öhrvall, A., & Rosenbaum, P. (2007) · Developmental Medicine & Child Neurology · doi:10.1111/j.1469-8749.2006.tb01313.x · Cited by 542
Basal Ganglia Volumes in Patients With Gilles de la Tourette Syndrome
Peterson, B. S., Thomas, P., Kane, M. J., Scahill, L., Zhang, H., Bronen, R., King, R. A., Leckman, J. F., & Staib, L. (2003) · Archives of General Psychiatry · doi:10.1001/archpsyc.60.4.415 · Cited by 527
The Pediatric Anxiety Rating Scale (PARS): development and psychometric properties
P, T. R. U. O. P. (2002) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200209000-00006 · Cited by 532
Moher, D., Fortin, P. R., Jadad, A. R., Jüni, P., Klassen, T. P., LeLorier, J., Liberati, A., Penna, A., & Linde, K. (1996) · The Lancet · doi:10.1016/s0140-6736(96)90538-3 · Cited by 539
Personality Judgments Based on Physical Appearance
Naumann, L. P., Vazire, S., Rentfrow, P. J., & Gosling, S. D. (2009) · Personality and Social Psychology Bulletin · doi:10.1177/0146167209346309 · Cited by 535
Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., Lenderking, W. R., Cella, D. F., & Basch, E. (2008) · Value in Health · doi:10.1111/j.1524-4733.2008.00470.x · Cited by 550
Generalist Genes and Learning Disabilities.
Plomin, R., & Kovas, Y. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.4.592 · Cited by 534
Comparability of telephone and face-to-face interviews in assessing axis I and II disorders
Rohde, P., Lewinsohn, P. M., & Seeley, J. R. (1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.11.1593 · Cited by 522
Objective Light-Intensity Physical Activity Associations With Rated Health in Older Adults
Buman, M. P., Hekler, E. B., Haskell, W. L., Pruitt, L., Conway, T. L., Cain, K. L., Sallis, J. F., Saelens, B. E., Frank, L. D., & King, A. C. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq249 · Cited by 528
Is it possible to be schizophrenic yet neuropsychologically normal?
Palmer, B. W., Heaton, R. K., Paulsen, J. S., Kuck, J., Braff, D., Harris, M. J., Zisook, S., & Jeste, D. V. (1997) · Neuropsychology · doi:10.1037//0894-4105.11.3.437 · Cited by 529
Authentic Leadership and Positive Psychological Capital
Clapp‐Smith, R., Vogelgesang, G. R., & Avey, J. B. (2008) · Journal of Leadership & Organizational Studies · doi:10.1177/1548051808326596 · Cited by 551
Kappa coefficients in medical research
KRAEMER, H. C., Periyakoil, V. S., & Noda, A. M. (2002) · Statistics in Medicine · doi:10.1002/sim.1180 · Cited by 547
Morosini, P. L., Magliano, L., Brambilla, L., Ugolini, S., & Pioli, R. (2000) · Acta Psychiatrica Scandinavica · doi:10.1034/j.1600-0447.2000.101004323.x · Cited by 1,097
Mintken, P. E., Glynn, P., & Cleland, J. A. (2009) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2008.12.015 · Cited by 600
The Rowland Universal Dementia Assessment Scale (RUDAS): a multicultural cognitive assessment scale
Storey, J. E., Rowland, J. T. J., Conforti, D. A., & Dickson, H. G. (2004) · International Psychogeriatrics · doi:10.1017/s1041610204000043 · Cited by 509
Kring, A. M., Smith, D. A., & Neale, J. M. (1994) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.66.5.934 · Cited by 525
Foa, E. B., Kozak, M. J., Goodman, W. K., Hollander, E., Jenike, M. A., & Rasmussen, S. A. (1995) · American Journal of Psychiatry · doi:10.1176/ajp.152.1.90 · Cited by 521
Uswatte, G., Taub, E., Morris, D. M., Vignolo, M. J., & McCulloch, K. L. (2005) · Stroke · doi:10.1161/01.str.0000185928.90848.2e · Cited by 519
THE INFLUENCE OF ACTIVISM BY INSTITUTIONAL INVESTORS ON R&D.
David, P., Hitt, M. A., & Gimeno, J. (2001) · Academy of Management Journal · doi:10.2307/3069342 · Cited by 521
Genetic Heterogeneity Between the Three Components of the Autism Spectrum: A Twin Study
RONALD, A., HAPPÉ, F., BOLTON, P., BUTCHER, L. M., PRICE, T. S., WHEELWRIGHT, S., BARON-COHEN, S., & PLOMIN, R. (2006) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000215325.13058.9d · Cited by 508
A disagreement about within-group agreement: disentangling issues of consistency versus consensus
Kozlowski, S. W., & Hattrup, K. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.2.161 · Cited by 514
Reliability of Measurements of Cervical Spine Range of Motion—Comparison of Three Methods
Youdas, J. W., Carey, J. R., & Garrett, T. R. (1991) · Physical Therapy · doi:10.1093/ptj/71.2.98 · Cited by 514
Psychometric considerations in evaluating health-related quality of life measures
Hays, R. D., Anderson, R., & Revicki, D. (1993) · Quality of Life Research · doi:10.1007/bf00422218 · Cited by 513
Adjunctive non-pharmacological analgesia for invasive medical procedures: a randomised trial
Lang, E. V., Benotsch, E. G., Fick, L. J., Lutgendorf, S., Berbaum, M. L., Berbaum, K. S., Logan, H., & Spiegel, D. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02162-0 · Cited by 513
Kennedy, D. M., Stratford, P. W., Wessel, J., Gollish, J. D., & Penney, D. (2005) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-6-3 · Cited by 512
BI-RADS Lexicon for US and Mammography: Interobserver Variability and Positive Predictive Value
Lazarus, E., Mainiero, M. B., Schepps, B., Koelliker, S. L., & Livingston, L. S. (2006) · Radiology · doi:10.1148/radiol.2392042127 · Cited by 507
A Note On "A Concordance Correlation Coefficient to Evaluate Reproducibility"
Nickerson, C. A. E. (1997) · Biometrics · doi:10.2307/2533516 · Cited by 517
Walton, D. M., MacDermid, J. C., Nielson, W. R., Teasell, R. W., Chiasson, M., & Brown, L. (2011) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2011.3666 · Cited by 506
Reliability from α to ω: A tutorial.
Revelle, W., & Condon, D. M. (2019) · Psychological Assessment · doi:10.1037/pas0000754 · Cited by 569
McQueen, F. M., Stewart, N. R., Crabbe, J. P., Robinson, E. T., Yeoman, S., Tan, P., & McLean, L. (1998) · Annals of the Rheumatic Diseases · doi:10.1136/ard.57.6.350 · Cited by 507
Moher, D., Pham, B., Lawson, M., & Klassen, T. (2003) · Health Technology Assessment · doi:10.3310/hta7410 · Cited by 500
Yule, S., Flin, R. H., Maran, N., ROWLEY, D. I., Youngson, G. G., & Paterson‐Brown, S. (2008) · World Journal of Surgery · doi:10.1007/s00268-007-9320-z · Cited by 507
Risk Factors for Stress Fractures in Track and Field Athletes
Bennell, K. L., Malcolm, S. A., Thomas, S. A., Reid, S. J., Brukner, P. D., Ebeling, P. R., & Wark, J. D. (1996) · The American Journal of Sports Medicine · doi:10.1177/036354659602400617 · Cited by 506
Valstar, M., Gratch, J. S., Schuller, B. W., Ringeval, F., Lalanne, D., Torres, M. T., Scherer, S. J., Stratou, G., Cowie, R., & Pantić, M. (2016) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2988257.2988258 · Cited by 540
Good debt or bad debt: Detecting semantic orientations in economic texts
Malo, P., Sinha, A., Korhonen, P., Wallenius, J., & Takala, P. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23062 · Cited by 513
The Effect of Time on Static Stretch on the Flexibility of the Hamstring Muscles
Bandy, W. D., & Irion, J. M. (1994) · Physical Therapy · doi:10.1093/ptj/74.9.845 · Cited by 504
The Geometry of the Tibial Plateau and Its Influence on the Biomechanics of the Tibiofemoral Joint
Hashemi, J., Chandrashekar, N., Gill, B., et al. (2008) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.g.01358 · Cited by 503
Dreu, C. K. W. D. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.3.628 · Cited by 533
Kennedy, K. M., & Raz, N. (2009) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2009.01.001 · Cited by 475
Validation of the Turkish Version of the Oswestry Disability Index for Patients With Low Back Pain
Yakut, E., Düger, T., Öksüz, Ç., Yörükan, S., Üreten, K., Turan, D., Frat, T., Kiraz, S., Kırdı, N., Kayhan, H., et al. (2004) · Spine · doi:10.1097/01.brs.0000113869.13209.03 · Cited by 495
Legro, M. W., Reiber, G. D., Smith, D. G., del Aguila, M., Larsen, J. A., & Boone, D. A. (1998) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(98)90090-9 · Cited by 507
Stewart, R. E., & Chambless, D. L. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0016032 · Cited by 483
The Repeatability of Clinical Measurements of Dry Eye
Nichols, K. K., Mitchell, G. L., & Zadnik, K. (2004) · Cornea · doi:10.1097/00003226-200404000-00010 · Cited by 504
A robust visual method for assessing the relative performance of edge-detection algorithms
Heath, M. D., Sarkar, S., Sanocki, T., et al. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.643893 · Cited by 494
Zhao, Y., Li, M., Konaté, M. M., Chen, L., Das, B., Karlovich, C., Williams, P. M., Evrard, Y. A., Doroshow, J. H., & McShane, L. M. (2021) · Journal of Translational Medicine · doi:10.1186/s12967-021-02936-w · Cited by 517
Effects of task performance and contextual performance on systemic rewards.
Scotter, J. R. V., Motowidlo, S. J., & Cross, T. C. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.4.526 · Cited by 495
THE MUTUAL INTELLIGIBILITY OF L2 SPEECH
Munro, M. J., Derwing, T. M., & Morton, S. L. (2006) · Studies in Second Language Acquisition · doi:10.1017/s0272263106060049 · Cited by 489
Analysis and Comparison of the Psychometric Properties of Three Balance Measures for Stroke Patients
Mao, H., Hsueh, I., Tang, P., Sheu, C., & Hsieh, C. (2002) · Stroke · doi:10.1161/01.str.0000012516.63191.c5 · Cited by 488
ASSESSING MANAGERIAL DISCRETION ACROSS INDUSTRIES: A MULTIMETHOD APPROACH.
Hambrick, D. C., & Abrahamson, E. (1995) · Academy of Management Journal · doi:10.2307/256864 · Cited by 485
Roos, E. M. (1999) · Scandinavian Journal of Rheumatology · doi:10.1080/03009749950155562 · Cited by 486
Fauth, B., Decristan, J., Decker, A.-T., Büttner, G., Hardy, I., Klieme, E., & Kunter, M. (2019) · Teaching and Teacher Education · doi:10.1016/j.tate.2019.102882 · Cited by 484
An empirical study of employee loyalty, service quality and firm performance in the service industry
Yee, R. W. Y., Yeung, A. C. L., & Cheng, T. C. E. (2009) · International Journal of Production Economics · doi:10.1016/j.ijpe.2009.10.015 · Cited by 515
Delphi as a method to establish consensus for diagnostic criteria
Graham, B. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00211-7 · Cited by 481
Kamdar, D., & van Dyne, L. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.5.1286 · Cited by 522
Kellner, C. H., Knapp, R. G., Petrides, G., Rummans, T. A., Husain, M. M., Rasmussen, K., Mueller, M., Bernstein, H. J., O’Connor, K., Smith, G., et al. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.12.1337 · Cited by 471
The Case for Using the Repeatability Coefficient When Calculating Test–Retest Reliability
Vaz, S. M. A., Falkmer, T., Passmore, A. E., Parsons, R. W., & Andreou, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073990 · Cited by 476
The changing signs in the relationships among self-efficacy, personal goals, and performance.
Vancouver, J. B., Thompson, C. M., & Williams, A. A. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.4.605 · Cited by 474
The Science of Reviewing Researcha
OXMAN, A. D., & GUYATT, G. H. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26342.x · Cited by 472
Recommendations for reporting the results of studies of instrument and scale development and testing
Streiner, D. L., & Kottner, J. (2014) · Journal of Advanced Nursing · doi:10.1111/jan.12402 · Cited by 463
Carroll, K. M., Ball, S. A., Nich, C., Martino, S., Frankforter, T. L., Farentinos, C., Kunkel, L. E., Mikulich‐Gilbertson, S. K., Morgenstern, J., Obert, J. L., et al. (2005) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2005.08.002 · Cited by 473
Statistical evaluation of agreement between two methods for measuring a quantitative variable
Lee, J., Koh, D. S. Q., & Ong, C. N. (1989) · Computers in Biology and Medicine · doi:10.1016/0010-4825(89)90036-x · Cited by 473
Relationships between musical structure and psychophysiological measures of emotion.
Gomez, P., & Danuser, B. (2007) · Emotion · doi:10.1037/1528-3542.7.2.377 · Cited by 471
360° feedback: Accuracy, reactions, and perceptions of usefulness.
Brett, J. F., & ATWATER, L. E. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.5.930 · Cited by 505
Brain anomalies in children exposed prenatally to a common organophosphate pesticide
Rauh, V., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X., Liu, J., Barr, D. B., Slotkin, T. A., & Peterson, B. S. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1203396109 · Cited by 477
Pleasure-arousal theory and the intensity of emotions.
Reisenzein, R. (1994) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.67.3.525 · Cited by 486
Cook, D. A., Bordage, G., & Schmidt, H. G. (2008) · Medical Education · doi:10.1111/j.1365-2923.2007.02974.x · Cited by 472
Craske, M. G., Meuret, A. E., Ritz, T., Treanor, M., Dour, H., & Rosenfield, D. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000396 · Cited by 451
Fulmer, I. S., Gerhart, B. A., & SCOTT, K. S. (2003) · Personnel Psychology · doi:10.1111/j.1744-6570.2003.tb00246.x · Cited by 474
Abusive supervision and subordinates’ organization deviance
Tepper, B. J., Henle, C. A., Lambert, L. S., Giacalone, R. A., & Duffy, M. K. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.4.721 · Cited by 468
PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction
Rasouli, A., Kotseruba, I., Kunić, T., & Tsotsos, J. K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00636 · Cited by 468
Knowledge management strategy and its link to knowledge creation process
Choi, B., & Lee, H. (2002) · Expert Systems with Applications · doi:10.1016/s0957-4174(02)00038-6 · Cited by 466
Sample size formulas for estimating intraclass correlation coefficients with precision and assurance
Zou, G. (2012) · Statistics in Medicine · doi:10.1002/sim.5466 · Cited by 473
Dent, E. B., Higgins, M. E., & Wharff, D. M. (2005) · The Leadership Quarterly · doi:10.1016/j.leaqua.2005.07.002 · Cited by 502
Subjective health complaints in adolescence—Reliability and validity of survey methods
HAUGLAND, S., & Wold, B. (2001) · Journal of Adolescence · doi:10.1006/jado.2000.0393 · Cited by 465
Too Proud to Let Go: Narcissistic Entitlement as a Barrier to Forgiveness.
Exline, J. J., Baumeister, R. F., Bushman, B. J., Campbell, W. K., & Finkel, E. J. (2004) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.87.6.894 · Cited by 465
Gunning‐Dixon, F. M., & Raz, N. (2003) · Neuropsychologia · doi:10.1016/s0028-3932(03)00129-5 · Cited by 483
Neuroanatomical correlates of cognitive aging: Evidence from structural magnetic resonance imaging.
Raz, N., Gunning-Dixon, F. M., Head, D., Dupuis, J. H., & Acker, J. D. (1998) · Neuropsychology · doi:10.1037/0894-4105.12.1.95 · Cited by 454
Gender differences in pressure pain threshold in healthy humans
Chesterton, L. S., Barlas, P., Foster, N. E., Baxter, D. G., & Wright, C. C. (2003) · Pain · doi:10.1016/s0304-3959(02)00330-5 · Cited by 460
e-Perceptions: Personality Impressions Based on Personal Websites.
Vazire, S., & Gosling, S. D. (2004) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.87.1.123 · Cited by 460
Promoting self-regulation through school-based martial arts training.
Lakes, K. D., & Hoyt, W. T. (2004) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2004.04.002 · Cited by 463
Uswatte, G., Taub, E., Morris, D. M., Light, K. E., & Thompson, P. A. (2006) · Neurology · doi:10.1212/01.wnl.0000238164.90657.c2 · Cited by 459
Riemann, R., Angleitner, A., & Strelau, J. (1997) · Journal of Personality · doi:10.1111/j.1467-6494.1997.tb00324.x · Cited by 457
Klässbo, M., Larsson, E., & Mannevik, E. (2003) · Scandinavian Journal of Rheumatology · doi:10.1080/03009740310000409 · Cited by 454
Stratford, P. W., & Goldsmith, C. H. (1997) · Physical Therapy · doi:10.1093/ptj/77.7.745 · Cited by 456
The impact of employee satisfaction on quality and profitability in high‐contact service industries
Yee, R. W. Y., Yeung, A. C. L., & Cheng, T. C. E. (2008) · Journal of Operations Management · doi:10.1016/j.jom.2008.01.001 · Cited by 490
Swami, V., & Barron, D. (2019) · Body Image · doi:10.1016/j.bodyim.2018.08.014 · Cited by 454
Regional Brain Volumes and Their Later Neurodevelopmental Correlates in Term and Preterm Infants
Peterson, B. S., Anderson, A. W., Ehrenkranz, R. A., Staib, L. H., Tageldin, M., Colson, E. R., Gore, J. C., Duncan, C. C., Makuch, R., & Ment, L. R. (2003) · PEDIATRICS · doi:10.1542/peds.111.5.939 · Cited by 451
A comparison of different indices of responsiveness
Wright, J. G., & Young, N. L. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00373-3 · Cited by 449
Feasibility and inter-rater reliability of the ICU Mobility Scale
Hodgson, C. L., Needham, D. M., Haines, K. J., Bailey, M., Ward, A., Harrold, M., Young, P. J., Zanni, J. M., Buhr, H., Higgins, A. M., et al. (2013) · Heart & Lung · doi:10.1016/j.hrtlng.2013.11.003 · Cited by 450
Reliability Studies of Psychiatric Diagnosis
Grove, W. M. (1981) · Archives of General Psychiatry · doi:10.1001/archpsyc.1981.01780290042004 · Cited by 450
Narrow, W. E., Clarke, D. E., Kuramoto, S. J., Kraemer, H. C., Kupfer, D. J., Greiner, L., & Regier, D. A. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12071000 · Cited by 446
Linear mixed-effects modeling approach to FMRI group analysis
Chen, G., Saad, Z. S., Britton, J. C., Pine, D. S., & Cox, R. W. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.01.047 · Cited by 440
Braun, U., Plichta, M. M., Esslinger, C., Sauer, C., Haddad, L., Grimm, O., Mier, D., Mohnke, S., Heinz, A., Erk, S., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.044 · Cited by 424
An Empirical Analysis of the Antecedents and Performance Consequences of Using the Moodle Platform
Huang, C., Wang, Y.-M., Wu, T.-W., & Wang, P. (2013) · International Journal of Information and Education Technology · doi:10.7763/ijiet.2013.v3.267 · Cited by 456
Generalizability studies of the Global Assessment of Functioning–Split version
Pedersen, G., Hagtvet, K. A., & Karterud, S. (2006) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2006.03.008 · Cited by 446
Efficacy of psychotherapeutic interventions to promote forgiveness: A meta-analysis.
Wade, N. G., Hoyt, W. T., Kidwell, J. E. M., & Worthington, E. L. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0035268 · Cited by 497
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
