Cited by 24,532Reliability and Agreement in MeasurementPsychometric Methodologies and TestingMultidisciplinary Science and Engineering Research

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.

Intraclass correlations: Uses in assessing rater reliability.

Shrout, Patrick E., Fleiss, Joseph L.·1979·Psychological Bulletin

DOI 10.1037/0033-2909.86.2.420

Open this paper in CoChat — free account

Abstract

Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.

Integrity signals

Registry

Crossref · 10.1037/0033-2909.86.2.420

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Reviews: Lord, Frederic M., and Novick, Melvin R. (with contributions by Allan Birnbaum). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley, 1968. xii + 568 pp. $14.95.
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Reviews: Lord, Frederic M., and Novick, Melvin R. (with contributions by Allan Birnbaum). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley, 1968. xii + 568 pp. $14.95.
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Statistical Methods -6th ed.-
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
The Analysis of Variance
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953

Show all 20 references

Sorted by citations

Items 22,801–22,900 of 23,883

Additional file 2 of Kinematic parameters obtained with the ArmeoSpring for upper-limb assessment after stroke: a reliability and learning effect study for guiding parameter use
Brihmat, N., Loubinoux, I., Castel-Lacanal, E., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13024287
Additional file 1 of Psychometric evaluation of an electronic Asthma Symptom Diary for young children
Williams, V. S. L., DeMuro, C., Clark, M., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26633910
Additional file 2 of Knee osteoarthritis pre-screening questionnaire (KOPS): cross-cultural adaptation and measurement properties of the brazilian version—KOPS Brazilian version
Trombini-Souza, F., Macedo, I., Yázigi, F., et al. (2022) · Figshare · doi:10.6084/m9.figshare.21506657
Additional file 5 of Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies
Ori, A. P. S., Lu, A. T., Horvath, S., & Ophoff, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21393676
Additional file 1 of Correlation of patient-reported numbness around surgical scars with patient-reported outcome measures and joint awareness after knee replacement: a cohort study
Itoh, M., Itou, J., Kuwashima, U., et al. (2022) · Figshare · doi:10.6084/m9.figshare.17799311
Additional file 1 of Non-local means based Rician noise filtering for diffusion tensor and kurtosis imaging in human brain and spinal cord
Zhang, Z., Vernekar, D., Qian, W., & Kim, M. (2021) · Figshare · doi:10.6084/m9.figshare.13669578
Additional file 1 of Psychometric validation of the Pyruvate Kinase Deficiency Diary and Pyruvate Kinase Deficiency Impact Assessment in adults in the phase 3 ACTIVATE trial
Andrae, D. A., Grace, R. F., Jewett, A., et al. (2023) · Figshare · doi:10.6084/m9.figshare.24540721
Additional file 6 of Targeted sequencing of NOTCH signaling pathway genes and association analysis of variants correlated with mandibular prognathism
Han, X., Xiong, X., Shi, X., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14685016
Additional file 2 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653909
Additional file 3 of The impact of inotersen on Neuropathy Impairment Score in patients with hereditary transthyretin amyloidosis with polyneuropathy
Yarlas, A. S., Lovley, A., Brown, D., Vera-Llonch, M., Khella, S. L., & Karam, C. (2023) · Figshare · doi:10.6084/m9.figshare.22618574
Additional file 2 of Validation and reliability of the Dutch version of the EORTC QLQ-BLM30 module for assessing the health-related quality of life of patients with muscle invasive bladder cancer
Ripping, T. M., Rammant, E., Witjes, J. A., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26558078
Additional file 1 of Assessment of affective dysregulation in children: development and evaluation of a semi-structured interview for parents and for children
Treier, A.-K., Labarga, S. Z., Ginsberg, C., Kohl, L. T., Görtz‐Dorten, A., Ravens‐Sieberer, U., Kaman, A., Banaschewski, T., Aggensteiner, P.-M., Hanisch, C., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26737125
Additional file 1 of Minimal detectable change of gait and balance measures in older neurological patients: estimating the standard error of the measurement from before-after rehabilitation data thanks to the linear mixed-effects models
Caronni, A., Picardi, M., Scarano, S., Rota, V., Guidali, G., Bolognini, N., & Corbo, M. (2024) · Figshare · doi:10.6084/m9.figshare.25530166
Additional file 1 of Increased lesion detectability in patients with locally advanced breast cancer—A pilot study using dynamic whole-body [18F]FDG PET/CT
Pedersen, M. A., Dias, A. H., Hjorthaug, K., et al. (2024) · Figshare · doi:10.6084/m9.figshare.25478776
Additional file 2 of Are exposure to health information and media health literacy associated with fruit and vegetable consumption?
Jindarattanaporn, N., Rittirong, J., Phulkerd, S., et al. (2023) · Figshare · doi:10.6084/m9.figshare.23962091
Additional file 3 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722179
Additional file 1 of Personalized radiomics signature to screen for KIT-11 mutation genotypes among patients with gastrointestinal stromal tumors: a retrospective multicenter study
Zhang, Q.-W., Zhang, R.-Y., Yan, Z.-B., et al. (2023) · Figshare · doi:10.6084/m9.figshare.24320013
Additional file 10 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653933
Additional file 1 of Assessing the psychosocial work environment in the health care setting: translation and psychometric testing of the French and Italian Copenhagen Psychosocial Questionnaires (COPSOQ) in a large sample of health professionals in Switzerland
Peter, K. A., Golz, C., Bürgin, R. A., Nübling, M., Voirol, C., Zürcher, S. J., & Hahn, S. (2022) · Figshare · doi:10.6084/m9.figshare.19726757
Additional file 8 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653927
Additional file 7 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653924
Additional file 1 of Psychometric validation and meaningful change thresholds of the Worst Itching Intensity Numerical Rating Scale for assessing itch in patients with chronic kidney disease-associated pruritus
Vernon, M., Swett, L. L., Speck, R. M., et al. (2021) · Figshare · doi:10.6084/m9.figshare.17598414
Additional file 4 of World Health Organisation Disability Assessment Schedule (WHODAS 2.0): development and validation of the Nigerian Igbo version in patients with chronic low back pain
Igwesi-Chidobe, C. N., Kitchen, S. S., Sorinola, I. O., & Godfrey, E. L. (2020) · Figshare · doi:10.6084/m9.figshare.13250892
Additional file 1 of Comparisons of clinical characteristics, brain MRI findings, and responses to epidural blood patch between spontaneous intracranial hypotension and post-dural puncture headache: retrospective study
Lee, G.-H., Kim, J., Kim, H.-W., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14881035
Additional file 1 of Gait apraxia evaluation in normal pressure hydrocephalus using inertial sensors. Clinical correlates, ventriculoperitoneal shunt outcomes, and tap-test predictive capacity
Ferrari, A., Milletti, D., Palumbo, P., Giannini, G., Cevoli, S., Magelli, E., Albini‐Riccioli, L., Mantovani, P., Cortelli, P., Chiari, L., et al. (2022) · Figshare · doi:10.6084/m9.figshare.20153139
Additional file 1 of Adaptation and multicentre validation of a patient-centred outcome scale for people severely ill with COVID (IPOS-COV)
Hocaoğlu, M., Murtagh, F. E. M., Walshe, C., Chambers, R. L., Maddocks, M., Sleeman, K., Oluyase, A. O., Dunleavy, L., Bradshaw, A., Bajwah, S., et al. (2023) · Figshare · doi:10.6084/m9.figshare.22619110
Additional file 1 of Quality appraisal of clinical practice guidelines addressing massage interventions using the AGREE II instrument
Fan, M., Liu, A., Lu, T., Zhou, X., Tian, C., Liu, B., Xie, Q., Cai, J., Yin, L., Ge, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.25374801
Additional file 5 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653918
Additional file 6 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722188
Additional file 9 of Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies
Ori, A. P. S., Lu, A. T., Horvath, S., & Ophoff, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21393688
Additional file 1 of Validation and reliability of the Dutch version of the EORTC QLQ-BLM30 module for assessing the health-related quality of life of patients with muscle invasive bladder cancer
Ripping, T. M., Rammant, E., Witjes, J. A., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26558075
Additional file 1 of Knee osteoarthritis pre-screening questionnaire (KOPS): cross-cultural adaptation and measurement properties of the brazilian version—KOPS Brazilian version
Trombini-Souza, F., Macedo, I., Yázigi, F., et al. (2022) · Figshare · doi:10.6084/m9.figshare.21506654
Additional file 5 of Cine MR feature tracking analysis for diagnosing thymic epithelial tumors: a feasibility study
Takumi, K., Nagano, H., Mukai, A., et al. (2023) · Figshare · doi:10.6084/m9.figshare.22729214
Additional file 1 of Comparability of a provisioned device versus bring your own device for completion of patient-reported outcome measures by participants with chronic obstructive pulmonary disease: quantitative study findings
Hudgens, S. A., Newton, L., Eremenco, S. L., Crescioni, M., Symonds, T. L., Griffiths, P. C. G., Reasner, D. S., Byrom, B., O’Donohoe, P., & Vallow, S. (2022) · Figshare · doi:10.6084/m9.figshare.21629259
Additional file 1 of Development and validation of the ND10 to measure neck-related functional disability
MacDermid, J. C., & Walton, D. M. (2022) · Figshare · doi:10.6084/m9.figshare.20138350
Additional file 3 of Reliability of tibiofemoral contact area and centroid location in upright, open MRI
Schmidt, A. M., Stockton, D. J., Hunt, M. A., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13310431
Additional file 4 of Targeted sequencing of NOTCH signaling pathway genes and association analysis of variants correlated with mandibular prognathism
Han, X., Xiong, X., Shi, X., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14685010
Additional file 14 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653948
Additional file 1 of Kinematic parameters obtained with the ArmeoSpring for upper-limb assessment after stroke: a reliability and learning effect study for guiding parameter use
Brihmat, N., Loubinoux, I., Castel-Lacanal, E., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13024284
Additional file 1 of Psychometric evaluation of the Urgency NRS as a new patient-reported outcome measure for patients with ulcerative colitis
Dubinsky, M. C., Shan, M., Delbecque, L., et al. (2022) · Figshare · doi:10.6084/m9.figshare.21510430
Additional file 2 of Only two subscales of the Coping Strategies Questionnaire are culturally relevant for people with chronic low back pain in Nigerian Igbo populations: a cross-cultural adaptation and validation study
Igwesi-Chidobe, C. N., Sorinola, I. O., & Godfrey, E. L. (2021) · Figshare · doi:10.6084/m9.figshare.16590380
Additional file 1 of Deep learning for semi-automated unidirectional measurement of lung tumor size in CT
Woo, M., Devane, A. M., Lowe, S. C., Lowther, E. L., & Gimbel, R. W. (2021) · Figshare · doi:10.6084/m9.figshare.14833273
Additional file 3 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.26649609
Additional file 8 of Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies
Ori, A. P. S., Lu, A. T., Horvath, S., & Ophoff, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21393685
Additional file 1 of Are exposure to health information and media health literacy associated with fruit and vegetable consumption?
Jindarattanaporn, N., Rittirong, J., Phulkerd, S., et al. (2023) · Figshare · doi:10.6084/m9.figshare.23962088
Additional file 1 of Predicting chronological age of 14 or 18 in adolescents: integrating dental assessments with machine learning
Shen, S., Guo, Y., Han, J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26704683
Additional file 1 of Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies
Ori, A. P. S., Lu, A. T., Horvath, S., & Ophoff, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21393664
Additional file 1 of Cross-cultural adaptation and validation of the Chinese version of the revised surveys on patient safety culture™ (SOPS®) hospital survey 2.0
Wu, Y., Hua, W., Zhu, D., Onishi, R., Yang, Y., & Hasegawa, T. (2023) · Figshare · doi:10.6084/m9.figshare.22614023
Additional file 1 of A person-reported cumulative social risk measure does not show bias by income and education
Jones, S. M. W., Briant, K. J., Doody, D. R., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26759294
Additional file 2 of Agreement between 24-hour urine and 24-hour food recall in measuring salt intake in primary school children in Australia
Santos, J. A., Bolton, D. K., Rosewarne, E., et al. (2022) · Figshare · doi:10.6084/m9.figshare.21564156
Additional file 3 of Development and validation of the ND10 to measure neck-related functional disability
MacDermid, J. C., & Walton, D. M. (2022) · Figshare · doi:10.6084/m9.figshare.20138356
Additional file 4 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.26649612
Additional file 15 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653951
FIGURES 20–21. Puliciphora females, abdominal tergites 4–6 in Rudimentary halteres of some flightless female Phoridae (Diptera) reassessed, and some taxonomic implications
Disney, R. H. L. (2005) · INFM-OAR (INFN Catania) · doi:10.5281/zenodo.169852
Additional file 1 of TPM, FPKM, or Normalized Counts? A Comparative Study of Quantification Measures for the Analysis of RNA-seq Data from the NCI Patient-Derived Models Repository
Zhao, Y., Li, M.-C., Konaté, M. M., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14826106
Additional file 4 of Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies
Ori, A. P. S., Lu, A. T., Horvath, S., & Ophoff, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21393673
Additional file 1 of A systematic review of tools designed for teacher proxy-report of children’s physical literacy or constituting elements
Essiet, I. A., Lander, N., Salmon, J., Duncan, M. J., Eyre, E., Ma, J., & Barnett, L. M. (2021) · Figshare · doi:10.6084/m9.figshare.16767712
Additional file 3 of Manual and automated analysis of atrophy patterns in dementia with Lewy bodies on MRI
Khadhraoui, E., Müller, S. J., Hansen, N., Riedel, C. H., Langer, P., Timäus, C., Wiltfang, J., Bouter, C., Lange, C., & Ernst, M. S. (2022) · Figshare · doi:10.6084/m9.figshare.19418607
Additional file 1 of Multidisciplinary digital competencies of pre-service vocational teachers
Roll, M. J. J., & Ifenthaler, D. (2021) · Figshare · doi:10.6084/m9.figshare.14091460
Additional file 3 of World Health Organisation Disability Assessment Schedule (WHODAS 2.0): development and validation of the Nigerian Igbo version in patients with chronic low back pain
Igwesi-Chidobe, C. N., Kitchen, S. S., Sorinola, I. O., & Godfrey, E. L. (2020) · Figshare · doi:10.6084/m9.figshare.13250889
Additional file 2 of Reliability of tibiofemoral contact area and centroid location in upright, open MRI
Schmidt, A. M., Stockton, D. J., Hunt, M. A., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13310425
Additional file 1 of Cross-sectoral inter-rater reliability of the clinical frailty scale – a Danish translation and validation study
Nissen, S. K., Fournaise, A., Lauridsen, J. T., Ryg, J., Nickel, C. H., Gudex, C., Brabrand, M., & Andersen‐Ranberg, K. (2020) · Figshare · doi:10.6084/m9.figshare.13185918
Additional file 7 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722191
Additional file 1 of Cross-cultural adaptation and validation of the German Central Sensitization Inventory (CSI-GE)
Klute, M., Laekeman, M., Kuss, K., Petzke, F. W., Dieterich, A. V., Leha, A., Neblett, R., Ehrhardt, S., Ulma, J., & Schäfer, A. G. M. (2021) · Figshare · doi:10.6084/m9.figshare.15253851
Additional file 1 of Comparison of spatio-temporal gait parameters between the GAITRite® platinum plus classic and the GAITRite® CIRFACE among older adults: a retrospective observational study
Sacco, G., Ben-Sadoun, G., Gautier, J., Simon, R., Goupil, M., Laureau, P., Terrien, J., & Annweiler, C. (2024) · Figshare · doi:10.6084/m9.figshare.26573494
Additional file 7 of Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies
Ori, A. P. S., Lu, A. T.-H., Horvath, S., & Ophoff, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21393682
Additional file 16 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653954
Additional file 1 of Evaluating the ability of a predictive vision-based machine learning model to measure changes in gait in response to medication and DBS within individuals with Parkinson’s disease
Sabo, A., Iaboni, A., Taati, B., Fasano, A. A., & Gorodetsky, C. (2024) · Figshare · doi:10.6084/m9.figshare.26652270
Additional file 8 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722194
Additional file 2 of A systematic review of tools designed for teacher proxy-report of children’s physical literacy or constituting elements
Essiet, I. A., Lander, N., Salmon, J., Duncan, M. J., Eyre, E., Ma, J., & Barnett, L. M. (2021) · Figshare · doi:10.6084/m9.figshare.16767715
Additional file 2 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.24751605
Additional file 1 of Excellent agreement of Norwegian trauma registry data compared to corresponding data in electronic patient records
Naberezhneva, N., Uleberg, O., Dahlhaug, M., Giil-Jensen, V., Ringdal, K. G., & Røise, O. (2024) · Figshare · doi:10.6084/m9.figshare.26619931
Additional file 1 of The association of conspiracy beliefs and the uptake of COVID-19 vaccination: a cross-sectional study
Kowalska‐Duplaga, K., & Duplaga, M. (2024) · Figshare · doi:10.6084/m9.figshare.26581577
Additional file 1 of Feasibility of an Audit and Feedback Intervention to Facilitate Journal Policy Change Towards Greater Promotion of Transparency and Openness in Sports Science Research
Hansford, H. J., Cashin, A. G., Bagg, M. K., Wewege, M. A., Ferraro, M. C., Kianersi, S., Mayo-Wilson, E., Grant, S., Toomey, E., Skinner, I., et al. (2022) · Figshare · doi:10.6084/m9.figshare.20445409
Additional file 1 of Quantitative analysis of the ACL and PCL using T1rho and T2 relaxation time mapping: an exploratory, cross-sectional comparison between OA and healthy control knees
Ranmuthu, C. D. S., Mackay, J., Crowe, V. A., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16909614
Additional file 1 of World Health Organisation Disability Assessment Schedule (WHODAS 2.0): development and validation of the Nigerian Igbo version in patients with chronic low back pain
Igwesi-Chidobe, C. N., Kitchen, S. S., Sorinola, I. O., & Godfrey, E. L. (2020) · Figshare · doi:10.6084/m9.figshare.13250883
Additional file 2 of Quality appraisal of clinical practice guidelines addressing massage interventions using the AGREE II instrument
Fan, M., Liu, A., Lu, T., Zhou, X., Tian, C., Liu, B., Xie, Q., Cai, J., Yin, L., Ge, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.25374856
Additional file 2 of Targeted sequencing of NOTCH signaling pathway genes and association analysis of variants correlated with mandibular prognathism
Han, X., Xiong, X., Shi, X., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14685004
Additional file 1 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653906
Additional file 1 of The Preventiometer - reliability of a cardiovascular multi-device measurement platform and its measurement agreement with a cohort study
Junge, M., Krüger, M., Wahner‐Roedler, D. L., Bauer, B. A., Dörr, M., Bahls, M., Chenot, J., Biffar, R., & Schmidt, C. O. (2024) · Figshare · doi:10.6084/m9.figshare.26584632
Additional file 1 of Development of a preliminary conceptual model of the patient experience of chronic kidney disease: a targeted literature review and analysis
Flythe, J. E., Karlsson, N., Sundgren, A. K., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14833458
Additional file 2 of Predicting incident radiographic knee osteoarthritis through quantitative meniscal lesion parameters: data from the osteoarthritis initiative
Bo, K., Xie, X., Liu, X., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26507299
Additional file 1 of Maternal depressive symptom trajectories and associations with child feeding
Fox, K., Hart, C. N., Phelan, S., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26736345
Additional file 3 of Diagnostic accuracy of the peripheral venous pressure variation induced by an alveolar recruitment maneuver to predict fluid responsiveness during high-risk abdominal surgery
Desebbe, O., Vallier, S., Gergelé, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26605927
Communication Accommodation in Text Messages: Exploring Liking, Power, and Sex as Predictors of TextismsPreprint
Miles, J., Giles, H., Dunbar, N. E., et al. (2017) · OSF Preprints (OSF Preprints)
The Role of the Therapeutic Alliance in Dropout in Cognitive Processing Therapy for Posttraumatic Stress Disorder
Shayani, D. R., Canale, C. A., LoSavio, S. T., Sloan, D. M., & Hayes, A. M. (2025) · Behavior Therapy · doi:10.1016/j.beth.2025.09.002
Intra- and inter-rater reliability of the ActivForce digital dynamometer for the isometric strength of the neck muscles in patients with chronic neck pain
Lazoura, E., Savva, C., Papacharalambous, C., Ploutarchou, G., Pavlou, K., Christofi, I., Rentzias, P., & Karagiannis, C. (2025) · European Journal of Physiotherapy · doi:10.1080/21679169.2025.2574674
Leistungssteuerung
Ferrauti, A., Schneider, C., & Wiewelhove, T. (2025) · Springer eBooks · doi:10.1007/978-3-662-69524-1_3
Service Climate and Employee Satisfaction in Linkage Research
Lundby, K., & Fenlason, K. (2000) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-60752-837-120251009
Multicentric harmonization of a 6-color flow cytometry panel for immunological monitoring of human whole blood and PBMC samples
Sorsa, V. L., Macchia, I., & Urbani, F. (2025) · Methods in cell biology · doi:10.1016/bs.mcb.2025.10.007
Analysis of student thinking and inter-rater reliability in IB-based mathematics extended-response assessment: Focusing on longitudinal data from Gyeonggi Province
Cho, Y., & Kwon, O. N. (2025) · The Mathematical Education · doi:10.63311/mathedu.25.6442
Agreement Between Self- and Caregiver-Report of Thought Disturbances in Adults with Williams Syndrome and Intellectual DisabilityPreprint
Vassall, S. G., Kittleson, A. R., Halverson, A. S., Schock, G. V., Leslie, E., Dykens, E., Roof, E., Sheffield, J. M., & Bress, K. S. (2025) · medRxiv · doi:10.64898/2025.12.15.25341842 · Cited by 1
Challenges in Small Sample Size Cluster Trials for Medication Adherence: Insights From TAKE ‐ IT ‐ TOO
Riddell, S. W., Platt, R. W., Dew, M. A., Dabbs, A. D., Dharnidharka, V. R., Knaüper, B., Mayersohn, G. S., Teoh, C. W., Phan, V., Blydt‐Hansen, T. D., et al. (2025) · Pediatric Transplantation · doi:10.1111/petr.70250
Disentangling Brain-Psychopathology Associations: A Systematic Evaluation of Transdiagnostic Latent Factor ModelsPreprint
Gell, M., Hoffmann, M. S., Moore, T. M., Nikolaidis, A., Gur, R. C., Salum, G. A., Milham, M. P., Langner, R., Müller, V. I., Eickhoff, S. B., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.21.695029
Spider Test Modified for Pickleball: Reliable, but Do Not Use It
Falknor, M. J., Martin, E., & Kim, S. B. (2025) · J — Multidisciplinary Scientific Journal · doi:10.3390/j9010001 · Cited by 1
Automatic Downstep in Gungbe: Acoustic Evidence from Tonal SequencesPreprint
SOVI-GUIDI, W. L. P., & Wen, X. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5967830
Precision, accuracy, and reliability of a threshold hunting method for transcranial magnetic stimulation
Shirota, Y., Otsuka, J., & Hamada, M. (2025) · Clinical Neurophysiology Practice · doi:10.1016/j.cnp.2025.12.008
Improving Distortion-Product Otoacoustic Emission Test-Retest Reliability With Forward-Pressure Level Calibration on Clinical Equipment
Miller, J. A. L., Reed, C. M., & Perez, Z. D. (2025) · Ear and Hearing · doi:10.1097/aud.0000000000001778
“Sexuality Assessment Beyond Sexual Function: an Integrative Review of the Available Instruments in Brazilian Portuguese”
Brasil, M. L. D. A., Figueira, G. L., Pires, P. O., Zakhour, S., Nardi, A. E., & Sardinha, A. (2025) · Sexuality Research and Social Policy · doi:10.1007/s13178-025-01266-6
Shoulder Structural Changes in Manual Wheelchair Users With Spinal Cord Injury: Cross-Sectional Ultrasound Study of Supraspinatus Tendon Thickness and Acromiohumeral Distance
Eyvaz, N., Adar, S., Akçin, A. İ., et al. (2025) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2025.12.010

Authors: Shrout, P. E., & Fleiss, J. L.
Affiliations: Columbia University (US)
ORCID: Shrout, Patrick E.
Venue: Psychological Bulletin
Publisher: American Psychological Association (APA)
Published: 1979
Volume: 86
Issue: 2
Pages: 420-428
ISSN: 0033-2909 (ISSN-L)
1939-1455
DOI: 10.1037/0033-2909.86.2.420
OpenAlex: W2141403362
Cited by: 24,532
References: 20
Citations by year: 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
FWCI: 13.77
Type: journal article
Language: en
Open access: closed
Locations: Psychological Bulletin (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
Keywords: intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
MeSH terms: Humans, Reproducibility of Results, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Intraclass correlations: Uses in assessing rater reliability.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks