The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Intraclass correlations: Uses in assessing rater reliability.
Abstract
Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As a result, the inter-rater agreement was quite high for categorical variables (mean Cohen's kappa = 0.73, SD = 0.13) and for continuous variables (mean intraclass correlation [26] = 0.80, SD = .14).”
From Social Relationships and Mortality Risk: A Meta-analytic Review (2010) · cites this paper as [26]
In the citing paper
“we used the methodology Shrout and Fleiss (1979) describe and computed the Shrout-Fleiss single ICC score agreement of .998, which is considered quite high (Koo and Li 2016).”
From Driving Brand Engagement Through Online Social Influencers: An Empirical Investigation of Sponsored Blogging Campaigns (2019) · cites this paper as Shrout and Fleiss (1979)
In the citing paper
“In contrast, Cohen's quadratically weighted kappa converges to two-factor intraclass correlation coefficients with random and fixed raters (i.e., the so-called ICC (2,1) and ICC(3,1) coefficients), where the items and raters are the two factors (Carrasco, 2010;Fleiss & Cohen, 1973;Janson & Olsson, 2001;Schuster, 2004;Schuster & Smith, 2005;Shrout & Fleiss, 1979;Warrens, 2014).”
From Quadratically Weighted Agreement Coefficients: Interpretations and Connections (2026) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“A sample of 11% of video data was double coded by the researcher and an experienced coder (the developer of CHIPPA). Inter‐rater reliability was based on intra‐class correlation (ICC) statistics from the mean rating for the two coders (k = 2) using a two‐way mixed effects consistency model (see Table 2 below) and resulted in good to excellent reliability across the CHIPPA subscales (Koo & Li, 2016”
From A plausible role of imagination in pretend play, counterfactual reasoning, and executive functions (2023) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“The results suggest that coders showed high agreement (intraclass correlation [2, 1] = 0.85; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“Interrater reliability of the pre- and posttreatment assessments was excellent (intraclass correlations [2, 1] = 0.97 and 0.96, respectively; Shrout & Fleiss, 1979).”
From Respiratory and cognitive mediators of treatment change in panic disorder: Evidence for intervention specificity. (2010) · cites this paper as Shrout & Fleiss, 1979
In the citing paper
“ICC decomposes observed variance into between-query variance (task difficulty) and within-query variance (agent inconsistency), highlighting whether reported results reflect true capability or measurement noise.”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“The ICC quantifies the proportion of total variance attributable to differences between tasks and can be understood as a function of both the difficulty of the dataset (variance between tasks) and the consistency of the agent (variance within tasks).”
From Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation (2025) · cites this paper as Shrout and Fleiss 1979
In the citing paper
“Reproducibility of the final placenta, fetus and amniotic fluid volume in millilitres was assessed using the ICC (2,1) [37], [38] for inter-observer repeatability between the semi-automated Random Walker results, regarded as the ground-truth used for performance evaluation, and the newly generated FCNN outputs.”
From Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment (2021) · cites this paper as [37]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Cited by
Sorted by citations
Items 22,301–22,400 of 23,883
Cha, Y., Chung, J. Y., Kim, J.-W., et al. (2024) · Bioengineering · doi:10.3390/bioengineering11040327
Radioproteomics stratifies molecular response to antifibrotic treatment in pulmonary fibrosisPreprint
Lauer, D., Magnin, C., Kolly, L. R., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.03.27.586923
Developing a Writing Rubric to Answer Research Questions (not for Grading!)
Lynch, J., Sorby, S., Aller, B. M., & Murphy, T. J. (2024) · ASEE North Central Section Conference · doi:10.18260/1-2--45609
Establishing the Job-Relatedness of Criminal Convictions
Ryan, A. M., & Ali, A. A. (2024) · Palgrave studies in equity, diversity, inclusion, and indigenization in business · doi:10.1007/978-3-031-54941-0_3
Yang, H., Chen, L., Cheng, Z., Yang, M., Wang, J., Lin, C.-H., Wang, Y., Huang, L., Chen, Y., Peng, S., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14331663
Yang, H., Chen, L., Cheng, Z., Yang, M., Wang, J., Lin, C.-H., Wang, Y., Huang, L., Chen, Y., Peng, S., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14331660
Min, S., Lee, S. H., Ga, I. H., et al. (2024) · Injury · doi:10.1016/j.injury.2024.111550
Li, J., Ou, J., Liu, Y., Shen, C., Chen, X., Li, Y., Zhao, J., Xu, J., Zhang, Y., & Wang, L. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-075748
The Significance of the Thoracic Spinal Multiple Segments During Different Pick-Object Approaches
Saad, W. A. A., Dzahir, M. A. M., Azaman, A. B., et al. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-99-8819-8_13
Carugati, F., Friard, O., Protopapa, E., Mancassola, C., Rabajoli, E., Gregorio, C. D., Valente, D., Ferrario, V., Cristiano, W., Raimondi, T., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4800104
Krupıć, D., Matijević, D., Šuvak, N., Ševerdija, D., & Maltar, J. (2024) · Research Square · doi:10.21203/rs.3.rs-4257623/v1
Reactions to Mega-Threats Among Members of the Harming Group:A Moral Cleansing ApproachPreprint
Motro, D., & Ellis, A. P. J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4803352
Ultrasound Examination of the Median Nerve at Different Levels: Inter-rater Reliability Study
Dede, B. T., Kılınç, F., Oğuz, M., et al. (2024) · Turkish Journal of Osteoporosis · doi:10.4274/tod.galenos.2023.30164
James, T. T., SelvaGanapathy, V., Dhargave, P., & Parekh, D. (2024) · Clinical Epidemiology and Global Health · doi:10.1016/j.cegh.2024.101624
Minaker, L. M., Menko, P., & Olona, D. (2024) · Public Health Nutrition · doi:10.1017/s1368980024000983
Reliability of a Test Method to Assess Ice Hockey Head Impacts
Nickel, L., Haid, D., Duncan, O., et al. (2024) · Lecture notes in mechanical engineering · doi:10.1007/978-3-031-58094-9_18
Comparison of Heard and Imagined Blends of Instrumental Dyads
Zhu, L., & McAdams, S. E. (2024) · Music & Science · doi:10.1177/20592043241246751
Towards concussion prevention in ice hockey: mechanical metamaterial liners and helmet assessment
Haid, D. (2024) · Sheffield Hallam University · doi:10.7190/shu-thesis-00602
Zhao, N., Qiao, Y., Yue, J., Jing, Y., Ge, Q., Zhang, M., Zhang, J., Yuan, Z., Xiang, Y., Wang, J., et al. (2024) · Ageing and Neurodegenerative Diseases · doi:10.20517/and.2023.31
Radiographical magnification of the shoulder region
Kratochvíl, A., Daniel, M., Fulín, P., et al. (2024) · Obere Extremität · doi:10.1007/s11678-024-00802-x
Test–retest reliability of meta analytic networks during naturalistic viewing
Kröll, J., Friedrich, P., Li, X., Nurislamova, Y., Kraljević, N., Geiger, A., Mans, J., Waite, L. K., Caspers, J., Qian, X., et al. (2026) · PLoS ONE · doi:10.1371/journal.pone.0346967
The Diagnostic Value of CT Radiomics in Small Anterior Mediastinal Nodules
王, 兆. (2024) · Advances in Clinical Medicine · doi:10.12677/acm.2024.1451535
Chun, E. H., Chung, M. H., Kim, J. E., Lee, H. S., Jo, Y., & Jun, J. H. (2024) · Scientific Reports · doi:10.1038/s41598-024-62355-x · Cited by 1
Ideational Slippage in Middle-Aged and Older Adults: A Preliminary Study
Arble, E. P., Steinert, S. W., Shankar, S., Cerjanic, A. M., Sutton, B. P., & Daugherty, A. M. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21060656
Castellote‐Caballero, Y., Aibar‐Almazán, A., Cabrera-Brito, M. E., García-Betancor, P. L., Carcelén‐Fraile, M. D. C., Lerma, L. L., & Hita‐Contreras, F. (2024) · JSES International · doi:10.1016/j.jseint.2024.05.003
Brooks, M., Parmar, N., & Kraak, W. J. (2024) · South African Journal of Sports Medicine · doi:10.17159/2078-516x/2024/v36i1a17109
Seeing Opposites: How Partisan Counter-Projection Fuels False PolarizationPreprint
Knowles, E. D., Mogami, M., & Yudkin, D. A. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/7kcnh
(2024) · Revista Española de Pedagogía · doi:10.22550/0034-9461.4085
Peris, M. T. M., Monfort-Pañego, M., Gómez-López, Á., & Olivares, E. M. (2024) · Revista Española de Pedagogía · doi:10.22550/0034-9461.4037
Exploring Interactive Gestures with Voice Assistant on HMDs in Social Situations
Yi, X., Kong, Y., Kang, X., et al. (2024) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw62533.2024.00322
Alqarni, A. A., Othman, R., Alabasi, U. M., Khan, F. R., Alhamed, A., Nogali, S., & Neblett, R. (2024) · Disability and Rehabilitation · doi:10.1080/09638288.2024.2362946
Riezler, S., & Hagmann, M. (2024) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-57065-0_3 · Cited by 1
Alrowayeh, H. N. (2024) · Clinical Neurophysiology Practice · doi:10.1016/j.cnp.2024.06.001
Mullin, H. A., Carpenter, C. M., Cwiek, A. P., Lan, G., Chase, S. O., Carter, E. A., Vervoordt, S. M., Rabinowitz, A. R., Venkatesan, U. M., & Hillary, F. G. (2024) · doi:10.31219/osf.io/c4s3n
Saeed, S. S., Shirmard, M. H., Baghban, A. A., Jutai, J. W., & Rezaee, M. (2024) · Assistive Technology · doi:10.1080/10400435.2024.2363383
Hofer, J., Busch, H., & Menon, J. A. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1355213 · Cited by 1
Blandin, R., Didone, V., Birkholz, P., et al. (2024) · doi:10.21437/issp.2024-4
Mrożek, K., Marchewka, J., Borowska, B., Budnik, A., Leszczyński, B., Wróbel, A., & Wronka, I. (2024) · Anthropological Review · doi:10.18778/1898-6773.87.2.03
Huber, S. D., & Alfuth, M. (2024) · Zeitschrift für Orthopädie und Unfallchirurgie · doi:10.1055/a-2331-1084
RetractedPowell, A., Mann, A. K., Rossmann, J. S., & Prabhu, R. (2024) · The Journal of Creative Behavior · doi:10.1002/jocb.682
Li, A. S., Myers, J., Stinnett, S. S., et al. (2024) · Ophthalmology Science · doi:10.1016/j.xops.2024.100568
Yogev, D., Madgar, O., Goldberg, T., et al. (2024) · Journal of Voice · doi:10.1016/j.jvoice.2024.06.011
Igarashi, Y., Marques, A. P., Andrade, K. F. A., Brito, A. L., Xavier, M. B., Souza, G. D. S., & Callegari, B. (2024) · Scientific Reports · doi:10.1038/s41598-024-68224-x
Zhan, Y., Dai, R. X., Li, F., Cheng, Z., Zhuo, Y., Shan, F., & Zhou, L. (2024) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-24-77
Costa, P. H. V., da Silva, L. V. T. T., de Jesus, T. P. D., Dario, D. M., Torriani‐Pasin, C., & Polese, J. C. (2024) · Journal of Bodywork and Movement Therapies · doi:10.1016/j.jbmt.2024.07.055
Bewertung von Tierwohlindikatoren für Mastschweine
Pfeifer, M. (2024) · eDiss (Georg-August-Universität Göttingen) · doi:10.53846/goediss-10641
McLaren, J., Makdissi, M., Hearps, S. J. C., & Davis, G. A. (2024) · Journal of Science and Medicine in Sport · doi:10.1016/j.jsams.2024.08.200
Is it normal to lead? Evaluating the role of superiority and similarity in leader emergence
Harms, P. D., Paterson, T. A., Wood, D., & Fezzey, T. N. A. (2024) · Frontiers in Organizational Psychology · doi:10.3389/forgp.2024.1357196 · Cited by 1
Reliability of a Cognitive Test Battery for Individuals with a Spinal Cord Injury
Emmenegger, A., Grossmann, F., Scherrer, S., & Perret, C. (2024) · Journal of Artificial Intelligence Machine Learning and Data Science · doi:10.51219/jih/claudio-perret/48
Reliability of Machine Learning in functional assessment in cardiac magnetic resonance imaging
Boccetti, D., Lamja, S., Palumbo, P., Barile, A., & Cesare, E. D. (2024) · Journal of Medical Imaging and Interventional Radiology · doi:10.1007/s44326-024-00032-z
Handout IPA Berbasis Problem (HIPAPe): Materi Bumi dan Tata Surya melalui Framework Design Thinking
Rahmawati, R., Sriwulan, W., & Samsudin, A. (2024) · Natural Jurnal Ilmiah Pendidikan IPA · doi:10.30738/natural.v11i1.17330
Takamiya, S., Echizenya, I., Yamazaki, K., et al. (2024) · World Neurosurgery · doi:10.1016/j.wneu.2024.08.167
Cleary, C. J., Rogers, K. B., Dombrowski, N., Vopat, L., Vopat, B. G., & Herda, A. A. (2024) · JOSPT open. · doi:10.2519/josptopen.2024.0051
Is a patient self‐recorded 6 minute walk test equivalent to a formally recorded 6 minute walk test
van Bavel, D., de Steiger, R. N., & McKenzie, D. P. (2024) · ANZ Journal of Surgery · doi:10.1111/ans.19213
Harris‐Hayes, M., Zorn, P., Steger-May, K., et al. (2024) · JOSPT open. · doi:10.2519/josptopen.2023.1020
Cognitive debriefing e fidedignidade do NOPPAIN-Br
Araujo, R. S., Pereira, L. V., de Araujo, D. S., et al. (2024) · Acta Paulista de Enfermagem · doi:10.37689/acta-ape/2024ao00000834
NOPPAIN-Br cognitive debriefing and reliability
Araujo, R. S., Pereira, L. V., de Araujo, D. S., et al. (2024) · Acta Paulista de Enfermagem · doi:10.37689/acta-ape/2024ao000008344
Niu, Y., Chen, Z., Jin, L., et al. (2024) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-024-07866-4
Public interest and seasonal peaks for wisdom teeth related web inquiries – A google trends analysis
Remschmidt, B., Muallah, D., Rieder, M., et al. (2024) · Health Informatics Journal · doi:10.1177/14604582241270759
Algorithmic assessment of drag on thermally cut sheet metal edges
Stahl, J., Zengl, S., Frommknecht, A., et al. (2024) · tm - Technisches Messen · doi:10.1515/teme-2024-0090
Impact or Responsibility? Giving Behavior in a Televised Natural ExperimentPreprint
Eberhardt, I., Smeets, P., van den Assem, M. J., & van Dolder, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4946619
Shorter, K. A., Tissera, K., Huynh, M., & Benson, A. C. (2024) · International Journal of Sports Science & Coaching · doi:10.1177/17479541241284714
Coakley, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4948019
Hao, Q., Yang, Y., Yang, S., Ding, Y., Li, C., Luo, G., & Zhang, X. (2024) · Research Square · doi:10.21203/rs.3.rs-4909934/v1
Pereira, S. S., & Šatas, S. (2024) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2024-002606 · Cited by 1
Intrarater and Interrater Reliability of the Quantification of Knee Cartilage MR Relaxation MetricsPreprint
Liao, T., Pang, Y., Yablon, C. M., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4926999/v1
Gao, J., Rose, A. M., Suh, I., et al. (2024) · Emerald Group Publishing Limited eBooks · doi:10.1108/s1475-148820240000027002
Measures of Interrater Agreement for Quantitative Scales Based on the Standard Deviation
Bove, G. (2024) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-031-65699-6_22
Weyrich, S., Scholz, V., Hetzke, L., et al. (2024) · PubMed · doi:10.25646/12536
Responses to observing others caught cheating: The role of schadenfreude
Motro, D., Perkins, B. G., & Ellis, A. P. J. (2024) · Journal of Management & Organization · doi:10.1017/jmo.2024.45 · Cited by 1
Kumar, A., & Upmanyu, A. (2024) · Physical Activity and Nutrition · doi:10.20463/pan.2024.0018
Petrovici, N., Pop, C., & Zbranca, R. (2024) · Empirical Studies of the Arts · doi:10.1177/02762374241291479
Validation of a Photonumeric Scale for Evaluation of Telangiectasia in Rosacea
Nguyen, L., Seeber, N., Baron, J. M., et al. (2024) · Journal of Cosmetic Dermatology · doi:10.1111/jocd.16593
Zhai, Z., Wang, Y., Li, Q., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-5309318/v1
Friedman, J., Sonay, T. B., Brown, A., McBurney, A., Cummings, K.-P., & Black, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5002708
The Influence of Perspective on Perception: Assessing Residue Across Planes of Videofluoroscopy
Jones‐Rastelli, R. B., Crossman, C., Price, D., Stal, F., & Molfenter, S. M. (2024) · The Laryngoscope · doi:10.1002/lary.31873
Structural and Functional Properties of Lower Extremity Tendons in Men
Comstock, B. A., Flanagan, S. D., Denegar, C. R., Newton, R. U., Häkkinen, K., Volek, J. S., Maresh, C. M., & Kraemer, W. J. (2024) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000004968
Thomas, L., Kulkarni, C., & Kuzhuppilly, N. I. R. (2024) · Taiwan Journal of Ophthalmology · doi:10.4103/tjo.tjo-d-24-00058
Looney, S. W., & Hagan, J. L. (2015) · Wiley eBooks · doi:10.1002/9781394321742.ref
Looney, S. W., & Hagan, J. L. (2015) · Wiley eBooks · doi:10.1002/9781394321742.oth1
Looney, S. W., & Hagan, J. L. (2015) · Wiley eBooks · doi:10.1002/9781394321742.ch5
Pilot Study on Postoperative Toric Intraocular Lens Alignment
Yang, B., Lai, C., Qin, Y., Lin, H.-L., Wang, S., Liao, H., & Zhang, H. (2024) · Journal of Ophthalmology · doi:10.1155/2024/1053914
Raju, V. R., & Devi, G. N. R. (2024) · IP Indian Journal of Neurosciences · doi:10.18231/j.ijn.2024.042
Arima, H., Yamato, Y., Shibata, Y., et al. (2024) · Journal of Orthopaedic Science · doi:10.1016/j.jos.2024.11.002
Taste Panellists’ Evaluations in Official Cheese Competitions: Analysis for Improvement Proposals
Arencibia, P. H., Saavedra, P., Carrascosa, C., et al. (2024) · Foods · doi:10.3390/foods13233769
Yousfi, S. (2024) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12364
Acetabular rim syndrome revisited
Morris, W. Z., Haider, S., Hinds, S. T., et al. (2024) · The Bone & Joint Journal · doi:10.1302/0301-620x.106b12.bjj-2024-0164.r1
Can bile excretion on Gd-EOB-MRI be used as a visual criterion for the hepatobiliary phase?
Nakamura, M., Takatsu, Y., Yoshizawa, M., et al. (2024) · Radiological Physics and Technology · doi:10.1007/s12194-024-00868-w
A Framework for Case Simulation Surveys: Advancing Clinical Diagnosis Decision-Making
Lim, J., & Zhao, M. (2024) · High Yield Medical Reviews · doi:10.59707/hymrytqn4575
Born-Again Multi-task Self-training for Multi-task Facial Emotion Recognition
Masumura, R., Takashima, A., Suzuki, S., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78444-6_7
González-Devesa, D., Ayán, C., Vila, P. V., et al. (2024) · Journal of Science in Sport and Exercise · doi:10.1007/s42978-024-00322-8
Metadata
- Authors
- Shrout, P. E., & Fleiss, J. L.
- Affiliations
- Columbia University (US)
- ORCID
- Shrout, Patrick E.
- Venue
- Psychological Bulletin
- Published
- 1979
- Volume
- 86
- Issue
- 2
- Pages
- 420-428
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2141403362
- Cited by
- 24,532
- References
- 20
- Citations by year
- 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
- FWCI
- 13.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
- Keywords
- intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
- MeSH terms
- Humans, Reproducibility of Results, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
