Cited by 24,532Reliability and Agreement in MeasurementPsychometric Methodologies and TestingMultidisciplinary Science and Engineering Research

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.

Intraclass correlations: Uses in assessing rater reliability.

Shrout, Patrick E., Fleiss, Joseph L.·1979·Psychological Bulletin

DOI 10.1037/0033-2909.86.2.420

Open this paper in CoChat — free account

Abstract

Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.

Integrity signals

Registry

Crossref · 10.1037/0033-2909.86.2.420

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Reviews: Lord, Frederic M., and Novick, Melvin R. (with contributions by Allan Birnbaum). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley, 1968. xii + 568 pp. $14.95.
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.3102/00028312006001112 · Cited by 8,013
Reviews: Lord, Frederic M., and Novick, Melvin R. (with contributions by Allan Birnbaum). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley, 1968. xii + 568 pp. $14.95.
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Statistical Methods -6th ed.-
Snedecor, G. W., & Cochran, W. G. (1967) · Cited by 6,573
The Analysis of Variance
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,206
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
The theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953

Show all 20 references

Sorted by citations

Items 22,701–22,800 of 23,883

Effectiveness of team-building and teamwork in real and virtual worlds
Ranade, R. S. (2009) · TigerPrints (Clemson University)
Schizophrenia-spectrum Behavior and Peer Responses to Individuals with Social Anhedonia
Baker, B. R. (2012) · University Libraries (University of Maryland)
Investigation of Optimal Cue to Instruction for Pelvic Floor Muscle Contraction in Women Using Ultrasound Imaging
Crotty, K. (2014) · University of Hertfordshire Research Archive (University of Hertfordshire) · doi:10.18745/th.15431
Influence of Scan Duration on the Reliability of Resting-State fMRI Regional Homogeneity
Xiao-tang, L., Zhou, J., & Liu, X. (2014) · Communications in computer and information science · doi:10.1007/978-3-662-45643-9_42
The patient's view of orthodontic training requirements and the influence of board certification or a master's degree on their choice for an orthodontist.
Trisler, J. (2012) · doi:10.18297/etd/1458
Perceptions of Psychological and Physiological Stress Responses: Process, Accuracy, and Measurement Convergence
Mason, A. E. (2012) · UA Campus Repository (The University of Arizona)
Investigation of Physical Activity Monitors and their Relationship to Traditional Physiological Measures during Human Exercise
Abrazado, M. (2012) · eScholarship (California Digital Library)
Ruqual: A System for Assessing Post-Editing
Housley, J. (2012) · ScholarsArchive (Brigham Young University)
In vitro comparison of gastric aspirate methods and feeding tube properties on the quantity and reliability of obtained aspirate volume
Ellis, B., & Rebecca, J. (2013)
Inter-rater reliability of the Norwegian translation of the Wolf Motor Function Test
Vatshaug, M. (2014)
Over structurering van beoordelingsmethoden voor open vragen
Frijns, P. H. A. M. (1993) · doi:10.26481/dis.19930304pf
DISPERSION PREDICTION MODELS IN NESTED DATA STRUCTURES: EXAMINING THE PERFORMANCE OF DISPERSION INDEXES IN POLYTOMOUS ITEMS
Yorio, P. L. (2014) · D-Scholarship@Pitt (University of Pittsburgh)
Additional file 1 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722173
Additional file 2 of Psychometric properties of the Japanese version of the Kansas City Cardiomyopathy Questionnaire in Japanese patients with chronic heart failure
Watanabe-Fujinuma, E., Origasa, H., Bamber, L., Roessig, L., Toyoda, T., Haga, Y., Gwaltney, C. J., & Pieske, B. M. (2020) · Figshare · doi:10.6084/m9.figshare.12671459
Additional file 1 of Assessment of the quality of end-of-life care: translation and validation of the German version of the “Care of the Dying Evaluation” (CODE-GER) - a questionnaire for bereaved relatives
Vogt, A., Stiel, S., Heckel, M., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12992081
Additional file 4 of Cross-cultural adaptation and validation of the German Central Sensitization Inventory (CSI-GE)
Klute, M., Laekeman, M., Kuss, K., Petzke, F. W., Dieterich, A. V., Leha, A., Neblett, R., Ehrhardt, S., Ulma, J., & Schäfer, A. G. M. (2021) · Figshare · doi:10.6084/m9.figshare.15253866
Additional file 1 of The reliability and validity of the Timed Up and Go test in patients ongoing or following lumbar spine surgery: a systematic review and meta-analysis
Özden, F. (2024) · Figshare · doi:10.6084/m9.figshare.25248901
Additional file 2 of Construct validity and reliability of the physical activity parenting questionnaire for children (PAP-C)
Laukkanen, A., Aunola, K., Korhonen, E., Barnett, L. M., & Sääkslahti, A. (2021) · Figshare · doi:10.6084/m9.figshare.14545111
Additional file 2 of Assessing the psychosocial work environment in the health care setting: translation and psychometric testing of the French and Italian Copenhagen Psychosocial Questionnaires (COPSOQ) in a large sample of health professionals in Switzerland
Peter, K. A., Golz, C., Bürgin, R. A., Nübling, M., Voirol, C., Zürcher, S. J., & Hahn, S. (2022) · Figshare · doi:10.6084/m9.figshare.19726760
Additional file 1 of A radiomics-based model on non-contrast CT for predicting cirrhosis: make the most of image data
Wang, J.-C., Fu, R., Tao, X.-W., Mao, Y.-F., Wang, F., Zhang, Z.-C., Yu, W.-W., Chen, J., He, J., & Sun, B.-C. (2020) · Figshare · doi:10.6084/m9.figshare.12972268
Additional file 1 of Normal sex and age-specific parameters in a multi-ethnic population: a cardiovascular magnetic resonance study of the Canadian Alliance for Healthy Hearts and Minds cohort
Luu, J. M., Gebhard, C., Ramasundarahettige, C. F., et al. (2022) · Figshare · doi:10.6084/m9.figshare.17799861
Additional file 2 of FDG PET versus CT radiomics to predict outcome in malignant pleural mesothelioma patients
Pavic, M., Bogowicz, M., Kraft, J., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650314
Additional file 1 of COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study
Mokkink, L. B., Boers, M., van der Vleuten, C. P. M., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2020) · Figshare · doi:10.6084/m9.figshare.13323854
Additional file 12 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653939
Additional file 2 of Looking for consistency in an uncertain world: test-retest reliability of neurophysiological and behavioral readouts in autism
Beker, S., Foxe, J. J., Venticinque, J., Bates, J. C., Ridgeway, E. M., Schaaf, R. C., & Molholm, S. (2021) · Figshare · doi:10.6084/m9.figshare.16714073
Additional file 5 of Targeted sequencing of NOTCH signaling pathway genes and association analysis of variants correlated with mandibular prognathism
Han, X., Xiong, X., Shi, X., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14685013
Additional file 10 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. T. (2021) · Figshare · doi:10.6084/m9.figshare.14722170
Additional file 13 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653942
Additional file 3 of A mixed-methods study of the effectiveness and perceptions of a course design institute for health science educators
Speer, J. E., Conley, Q., Thurber, D., Williams, B., Wasden, M., & Jackson, B. (2024) · Figshare · doi:10.6084/m9.figshare.26555159
Additional file 1 of Development of the Clinical Gestalt Assessment: a visual clinical global impression scale for Proteus syndrome
Ours, C., Hodges, M. B., Oden, N. L., Sapp, J. C., & Biesecker, L. G. (2022) · Figshare · doi:10.6084/m9.figshare.19646206
Additional file 2 of Validity, reliability, responsiveness, and clinically meaningful change threshold estimates of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Breast Cancer Symptom Index (NFBSI-16)
Clarke, N. A., Wong, B., Lawrance, R. M., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26777677
Additional file 1 of Intra- and inter-rater reliability of an electronic health record audit used in a chiropractic teaching clinic system: an observational study
Injeyan, H. S., Hogg‐Johnson, S., Abdulla, S., et al. (2021) · Figshare · doi:10.6084/m9.figshare.15071678
Additional file 2 of The impact of inotersen on Neuropathy Impairment Score in patients with hereditary transthyretin amyloidosis with polyneuropathy
Yarlas, A. S., Lovley, A., Brown, D., Vera-Llonch, M., Khella, S. L., & Karam, C. (2023) · Figshare · doi:10.6084/m9.figshare.22618571
Additional file 3 of Interrater reliability in the assessment of physiotherapy students
Gittinger, F. P., Lemos, M., Neumann, J. L., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19373547
Additional file 2 of Interrater reliability in the assessment of physiotherapy students
Gittinger, F. P., Lemos, M., Neumann, J. L., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19373544
Additional file 1 of The impact of inotersen on Neuropathy Impairment Score in patients with hereditary transthyretin amyloidosis with polyneuropathy
Yarlas, A. S., Lovley, A., Brown, D., Vera-Llonch, M., Khella, S. L., & Karam, C. (2023) · Figshare · doi:10.6084/m9.figshare.22618568
Additional file 2 of Manual and automated analysis of atrophy patterns in dementia with Lewy bodies on MRI
Khadhraoui, E., Müller, S. J., Hansen, N., Riedel, C. H., Langer, P., Timäus, C., Wiltfang, J., Bouter, C., Lange, C., & Ernst, M. S. (2022) · Figshare · doi:10.6084/m9.figshare.19418604
Additional file 1 of Phantom-based acquisition time and image reconstruction parameter optimisation for oncologic FDG PET/CT examinations using a digital system
Fragoso Costa, P., Jentzen, W., Brahmer, A., Mavroeidi, I.-A., Zarrad, F., Umutlu, L., Fendler, W. P., Rischpler, C., Herrmann, K., Conti, M., et al. (2022) · Figshare · doi:10.6084/m9.figshare.20506869
Additional file 4 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722182
Additional file 1 of Evaluation of symptoms in respiratory syncytial virus infection in adults: psychometric evaluation of the Respiratory Infection Intensity and Impact Questionnaire™ symptom scores
Osborne, R. H., Nelson, L. M., Fehnel, S. E., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26593310
Additional file 2 of Development and validation of the ND10 to measure neck-related functional disability
MacDermid, J. C., & Walton, D. M. (2022) · Figshare · doi:10.6084/m9.figshare.20138353
Additional file 4 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653915
Additional file 3 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653912
Additional file 1 of Comparing preprocessing strategies for 3D-Gene microarray data of extracellular vesicle-derived miRNAs
Takemoto, Y., Ito, D., Komori, S., Kishimoto, Y., Yamada, S., Hashizume, A., Katsuno, M., & Nakatochi, M. (2024) · Figshare · doi:10.6084/m9.figshare.26736783
Additional file 1 of Test-retest reliability and validity of vagally-mediated heart rate variability to monitor internal training load in older adults: a within-subjects (repeated-measures) randomized study
Manser, P., & de Bruin, E. D. (2024) · Figshare · doi:10.6084/m9.figshare.26739720
Additional file 11 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653936
Additional file 2 of Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722176
Additional file 2 of A radiomics-based model on non-contrast CT for predicting cirrhosis: make the most of image data
Wang, J.-C., Fu, R., Tao, X.-W., Mao, Y.-F., Wang, F., Zhang, Z.-C., Yu, W.-W., Chen, J., He, J., & Sun, B.-C. (2020) · Figshare · doi:10.6084/m9.figshare.12972271
Additional file 1 of Preselection of robust radiomic features does not improve outcome modelling in non-small cell lung cancer based on clinical routine FDG-PET imaging
Oliveira, C., Amstutz, F., Vuong, D., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16306024
Additional file 3 of A systematic review of tools designed for teacher proxy-report of children’s physical literacy or constituting elements
Essiet, I. A., Lander, N., Salmon, J., Duncan, M. J., Eyre, E., Ma, J., & Barnett, L. M. (2021) · Figshare · doi:10.6084/m9.figshare.16767718
Additional file 3 of Cross-cultural adaptation and validation of the German Central Sensitization Inventory (CSI-GE)
Klute, M., Laekeman, M., Kuss, K., Petzke, F. W., Dieterich, A. V., Leha, A., Neblett, R., Ehrhardt, S., Ulma, J., & Schäfer, A. G. M. (2021) · Figshare · doi:10.6084/m9.figshare.15253863
Additional file 9 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653930
Additional file 2 of World Health Organisation Disability Assessment Schedule (WHODAS 2.0): development and validation of the Nigerian Igbo version in patients with chronic low back pain
Igwesi-Chidobe, C. N., Kitchen, S. S., Sorinola, I. O., & Godfrey, E. L. (2020) · Figshare · doi:10.6084/m9.figshare.13250886
Additional file 1 of Mental health during ecological crisis: translating and validating the Hogg Eco-anxiety Scale for Argentinian and Spanish populations
de Pereira, A. R. Q., Loray, J. S. P., Moreno‐Poyato, A. R., Roldán-Merino, J. F., Botero, C., Bongiardino, L., Aufenacker, S. I., Stanley, S. K., Costa, T. F. O., Luís, S., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26711564
Additional file 4 of Reliability of tibiofemoral contact area and centroid location in upright, open MRI
Schmidt, A. M., Stockton, D. J., Hunt, M. A., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13310434
Additional file 1 of Predicting incident radiographic knee osteoarthritis through quantitative meniscal lesion parameters: data from the osteoarthritis initiative
Bo, K., Xie, X., Liu, X., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26507215
Additional file 1 of A head-to-head comparison of the EQ-5D-5L and 15D descriptive systems and index values in a general population sample
Nikl, A., Janssen, M. F., Brodszky, V., & Rencz, F. (2023) · Figshare · doi:10.6084/m9.figshare.22606741
Additional file 1 of Manual and automated analysis of atrophy patterns in dementia with Lewy bodies on MRI
Khadhraoui, E., Müller, S. J., Hansen, N., Riedel, C. H., Langer, P., Timäus, C., Wiltfang, J., Bouter, C., Lange, C., & Ernst, M. S. (2022) · Figshare · doi:10.6084/m9.figshare.19418601
Additional file 1 of Psychometric properties of the Alzheimer’s Disease Cooperative Study – Activities of Daily Living for Mild Cognitive Impairment (ADCS-MCI-ADL) scale: a post hoc analysis of the ADCS ADC-008 trial
Potashman, M. H., Pang, M., Tahir, M., et al. (2023) · Figshare · doi:10.6084/m9.figshare.22625293
Additional file 2 of Cross-cultural adaptation and validation of the German Central Sensitization Inventory (CSI-GE)
Klute, M., Laekeman, M., Kuss, K., Petzke, F. W., Dieterich, A. V., Leha, A., Neblett, R., Ehrhardt, S., Ulma, J., & Schäfer, A. G. M. (2021) · Figshare · doi:10.6084/m9.figshare.15253857
Additional file 1 of Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer’s dementia
Yeung, A., Iaboni, A., Rochon, E. A., Lavoie, M., Santiago, C., Yancheva, M., Novikova, J., Xu, M., Robin, J., Kaufman, L. D., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14737112
Additional file 1 of Only two subscales of the Coping Strategies Questionnaire are culturally relevant for people with chronic low back pain in Nigerian Igbo populations: a cross-cultural adaptation and validation study
Igwesi-Chidobe, C. N., Sorinola, I. O., & Godfrey, E. L. (2021) · Figshare · doi:10.6084/m9.figshare.16590377
Additional file 1 of Translation and validation of the German version of the Young Spine Questionnaire
Nyirö, L., Potthoff, T., Siegenthaler, M. H., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16435978
Additional file 3 of FDG PET versus CT radiomics to predict outcome in malignant pleural mesothelioma patients
Pavic, M., Bogowicz, M., Kraft, J., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650317
Additional file 1 of An evaluation of EQ-5D-3L health utility scores using five country-specific tariffs in a rural population aged 45–69 years in Hua county, Henan province, China
Wang, H., Cao, C., Guo, C., He, Y., Li, F., Xu, R., Liu, M., Liu, Z., Pan, Y., Liu, F., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650463
Additional file 1 of Validation and reliability of the Dutch version of the EORTC QLQ-NMIBC24 Questionnaire Module for patients with non-muscle-invasive bladder cancer
Ripping, T. M., Westhoff, E., Aaronson, N. K., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16650032
Additional file 2 of Excellent agreement of Norwegian trauma registry data compared to corresponding data in electronic patient records
Naberezhneva, N., Uleberg, O., Dahlhaug, M., Giil-Jensen, V., Ringdal, K. G., & Røise, O. (2024) · Figshare · doi:10.6084/m9.figshare.26619934
Additional file 6 of Shotgun metagenomics reveals interkingdom association between intestinal bacteria and fungi involving competition for nutrients
Xie, Z., Canalda-Baltrons, A., d’Enfert, C., & Manichanh, C. (2024) · Figshare · doi:10.6084/m9.figshare.26653921
Additional file 3 of Quality appraisal of clinical practice guidelines addressing massage interventions using the AGREE II instrument
Fan, M., Liu, A., Lu, T., Zhou, X., Tian, C., Liu, B., Xie, Q., Cai, J., Yin, L., Ge, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26690812
Additional file 1 of Looking for consistency in an uncertain world: test-retest reliability of neurophysiological and behavioral readouts in autism
Beker, S., Foxe, J. J., Venticinque, J., Bates, J. C., Ridgeway, E. M., Schaaf, R. C., & Molholm, S. (2021) · Figshare · doi:10.6084/m9.figshare.16714070
Additional file 4 of Interrater reliability in the assessment of physiotherapy students
Gittinger, F. P., Lemos, M., Neumann, J. L., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19373550
Additional file 1 of Construct validity and reliability of the physical activity parenting questionnaire for children (PAP-C)
Laukkanen, A., Aunola, K., Korhonen, E., Barnett, L. M., & Sääkslahti, A. (2021) · Figshare · doi:10.6084/m9.figshare.14545108
Additional file 2 of Diagnostic accuracy of the peripheral venous pressure variation induced by an alveolar recruitment maneuver to predict fluid responsiveness during high-risk abdominal surgery
Desebbe, O., Vallier, S., Gergelé, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26605924
Additional file 1 of Validity, reliability, responsiveness, and clinically meaningful change threshold estimates of the National Comprehensive Cancer Network-Functional Assessment of Cancer Therapy-Breast Cancer Symptom Index (NFBSI-16)
Clarke, N. A., Wong, B., Lawrance, R. M., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26777674
Additional file 4 of Diagnostic accuracy of the peripheral venous pressure variation induced by an alveolar recruitment maneuver to predict fluid responsiveness during high-risk abdominal surgery
Desebbe, O., Vallier, S., Gergelé, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26605930
Additional file 1 of Transcultural adaptation and validation of a French version of the University of California, Los Angeles geriatrics attitudes scale (UCLA-GAS-F)
Opsommer, E., Chevalley, O., Pegito, I., & Demeulenaere, P. (2021) · Figshare · doi:10.6084/m9.figshare.16584561
Additional file 1 of Diagnostic accuracy of the peripheral venous pressure variation induced by an alveolar recruitment maneuver to predict fluid responsiveness during high-risk abdominal surgery
Desebbe, O., Vallier, S., Gergelé, L., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26605921
Additional file 2 of A mixed-methods study of the effectiveness and perceptions of a course design institute for health science educators
Speer, J. E., Conley, Q., Thurber, D., Williams, B., Wasden, M., & Jackson, B. (2024) · Figshare · doi:10.6084/m9.figshare.26555156
FIGURES 39–41 in Redescription of Anovia circumclusa (Gorham) (Coleoptera: Coccinellidae: Noviini), with first description of the egg, larva, and pupa, and notes on adult intraspecific elytral pattern variation
Forrester, J. A., Vandenberg, N. J., & McHugh, J. V. (2009) · INFM-OAR (INFN Catania) · doi:10.5281/zenodo.187896
Additional file 1 of FDG PET versus CT radiomics to predict outcome in malignant pleural mesothelioma patients
Pavic, M., Bogowicz, M., Kraft, J., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650311
Additional file 1 of Quantification and reliability of [11C]VC - 002 binding to muscarinic acetylcholine receptors in the human lung — a test-retest PET study in control subjects
Cselényi, Z., Jučaitė, A., Kristensson, C., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12422435
Additional file 3 of Targeted sequencing of NOTCH signaling pathway genes and association analysis of variants correlated with mandibular prognathism
Han, X., Xiong, X., Shi, X., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14685007
Additional file 12 of Significant variation in the performance of DNA methylation predictors across data preprocessing and normalization strategies
Ori, A. P. S., Lu, A. T., Horvath, S., & Ophoff, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21393661
Additional file 1 of Reliability of tibiofemoral contact area and centroid location in upright, open MRI
Schmidt, A. M., Stockton, D. J., Hunt, M. A., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13310422
Additional file 1 of A mixed-methods study of the effectiveness and perceptions of a course design institute for health science educators
Speer, J. E., Conley, Q., Thurber, D., Williams, B., Wasden, M., & Jackson, B. (2024) · Figshare · doi:10.6084/m9.figshare.26555153
Additional file 2 of An evaluation of EQ-5D-3L health utility scores using five country-specific tariffs in a rural population aged 45–69 years in Hua county, Henan province, China
Wang, H., Cao, C., Guo, C., He, Y., Li, F., Xu, R., Liu, M., Liu, Z., Pan, Y., Liu, F., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650466
Additional file 2 of Normal sex and age-specific parameters in a multi-ethnic population: a cardiovascular magnetic resonance study of the Canadian Alliance for Healthy Hearts and Minds cohort
Luu, J. M., Gebhard, C., Ramasundarahettige, C. F., et al. (2022) · Figshare · doi:10.6084/m9.figshare.17799867
Additional file 2 of Translation and validation of the German version of the Young Spine Questionnaire
Nyirö, L., Potthoff, T., Siegenthaler, M. H., et al. (2021) · Figshare · doi:10.6084/m9.figshare.16435981
Additional file 1 of Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study
Yang, H., Chen, L., Cheng, Z., Yang, M., Wang, J., Lin, C., Wang, Y., Huang, L., Chen, Y., Peng, S., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14331657
Additional file 1 of Agreement between 24-hour urine and 24-hour food recall in measuring salt intake in primary school children in Australia
Santos, J. A., Bolton, D. K., Rosewarne, E., et al. (2022) · Figshare · doi:10.6084/m9.figshare.21564153
Additional file 1 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2023) · Figshare · doi:10.6084/m9.figshare.24751382
Additional file 1 of Risk factors and the CCTA application in patients with vulnerable coronary plaque in type 2 diabetes: a retrospective study
He, W., Fang, T., Fu, X., Lao, M., & Xiao, X. (2024) · Figshare · doi:10.6084/m9.figshare.26676803
Additional file 1 of Personality matters: exploring the relationship between personality and stress physiology in captive African lions
Vaz, J., Bartley, A., & Hunt, J. (2022) · Figshare · doi:10.6084/m9.figshare.19973323
Additional file 1 of Interrater reliability in the assessment of physiotherapy students
Gittinger, F. P., Lemos, M., Neumann, J. L., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19373541
Additional file 1 of Targeted sequencing of NOTCH signaling pathway genes and association analysis of variants correlated with mandibular prognathism
Han, X., Xiong, X., Shi, X., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14685001
Additional file 5 of Cross-cultural adaption and inter-rater reliability of the Swedish version of the updated clinical frailty scale 2.0
Olsson, H., Åhlund, K., Alfredsson, J., Andersson, D., Boström, A.-M., Guidetti, S. G. E., Prytz, M., & Ekerstad, N. (2024) · Figshare · doi:10.6084/m9.figshare.26649615
Additional file 7 of Targeted sequencing of NOTCH signaling pathway genes and association analysis of variants correlated with mandibular prognathism
Han, X., Xiong, X., Shi, X., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14685019
Additional file 1 of Detecting anxiety and depression among people with limited literacy living with chronic low back pain in Nigeria: adaptation and validation of the hospital anxiety and depression scale
Igwesi-Chidobe, C. N., Muomah, R. C., Sorinola, I. O., & Godfrey, E. L. (2021) · Figshare · doi:10.6084/m9.figshare.14558927
Additional file 1 of A novel semiautomated method for background activity and biological tumour volume definition to improve standardisation of 18F-FET PET imaging in glioblastoma
Brighi, C., Puttick, S. G., Li, S., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19128739
Additional file 1 of Psychometric properties of the Japanese version of the Kansas City Cardiomyopathy Questionnaire in Japanese patients with chronic heart failure
Watanabe-Fujinuma, E., Origasa, H., Bamber, L., Roessig, L., Toyoda, T., Haga, Y., Gwaltney, C. J., & Pieske, B. M. (2020) · Figshare · doi:10.6084/m9.figshare.12671456

Authors: Shrout, P. E., & Fleiss, J. L.
Affiliations: Columbia University (US)
ORCID: Shrout, Patrick E.
Venue: Psychological Bulletin
Publisher: American Psychological Association (APA)
Published: 1979
Volume: 86
Issue: 2
Pages: 420-428
ISSN: 0033-2909 (ISSN-L)
1939-1455
DOI: 10.1037/0033-2909.86.2.420
OpenAlex: W2141403362
Cited by: 24,532
References: 20
Citations by year: 2026: 1452 · 2025: 1085 · 2024: 998 · 2023: 1023 · 2022: 1083 · 2021: 1379 · 2020: 1189 · 2019: 1116 · 2018: 1178 · 2017: 1113 · 2016: 1106 · 2015: 1057 · 2014: 1172 · 2013: 1044 · 2012: 914
FWCI: 13.77
Type: journal article
Language: en
Open access: closed
Locations: Psychological Bulletin (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Multidisciplinary Science and Engineering Research
Keywords: intraclass correlation coefficient, rater reliability, inter-rater reliability, reliability coefficients, confidence intervals, multiple raters
MeSH terms: Humans, Reproducibility of Results, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Intraclass correlations: Uses in assessing rater reliability.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks