Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
The DOI matches its Crossref record; 125 of the 126 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, these two terms are not interchangeable and represent different notions of reliability (e.g., Kottner et al., 2011;Liao, Hunt, Chen, 2010;Stolarova, Wolf, Rinker, Brielmann, 2014).”
From Exploring Differences in Measurement and Reporting of Classroom Observation Inter-Rater Reliability (2018) · cites this paper as Kottner et al., 2011
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · cited 1×
“Although these guidelines are clearly arbitrary (Cicchetti et al., 2006; Landis and Koch, 1977) they have been widely adopted in the reporting of results.”
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
Quality criteria were proposed for measurement properties of health status questionnaires
Terwee, C. B., Bot, S. D. M., de Boer, M. R., van der Windt, D. A. W. M., Knol, D. L., Dekker, J., Bouter, L. M., & de Vet, H. C. W. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.012 · Cited by 10,764 · cited 1×
“80 are often used as the minimum standards for reliability coefficients, but this may be only sufficient for group-level comparisons or research purposes [12,58,108].”
Measuring agreement in method comparison studies
Bland, J. M., & Altman, D. G. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800204 · Cited by 9,619 · cited 1×
“The proposed measures of agreement include proportions of exact agreement [8,86,87], proportions of specific agreement [86,87], repeatability coefficients, and the graphical method proposed by Bland and Altman [88].”
Measuring agreement in method comparison studies
Bland, J. M., & Altman, D. G. (1999) · Statistical Methods in Medical Research · doi:10.1191/096228099673819272 · Cited by 9,619
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
Self-citation(2016) · Andalas University Repository (Andalas University) · Cited by 8,424
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903
Nursing Research Generating and Assessing Evidence for Nursing Practice
Timmins, F. (2013) · Nurse Education in Practice · doi:10.1016/j.nepr.2013.04.001 · Cited by 5,504
On the Theory of Scales of Measurement
Stevens, S. S. (1946) · Science · doi:10.1126/science.103.2684.677 · Cited by 4,761 · cited 1×
“Table 2 lists frequently applied statistical approaches as arranged by the level of measurement (Stevens, 1946) and by the use of reliability vs.”
Standards for Educational and Psychological Testing
(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n432 · Cited by 3,126
High agreement but low Kappa: I. the problems of two paradoxes
Feinstein, A. R., & Cicchetti, D. V. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90158-l · Cited by 3,034
Computing inter‐rater reliability and its variance in the presence of high agreement
Gwet, K. L. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711006x126600 · Cited by 2,167
Consensus methods: characteristics and guidelines for use
Fink, A., Kosecoff, J., Chassin, M., & Brook, R. H. (1984) · American Journal of Public Health · doi:10.2105/ajph.74.9.979 · Cited by 2,089 · cited 1×
“The development process contained elements of Glaser’s state-of-the-art method [27] and of the nominal group technique [28].”
Confidence intervals rather than P values: estimation rather than hypothesis testing.
Gardner, M. J., & Altman, D. G. (1986) · BMJ · doi:10.1136/bmj.292.6522.746 · Cited by 2,010
Health Measurement Scales--A Practical Guide to Their Development and Use
Waclawski, E. R. (2010) · Occupational Medicine · doi:10.1093/occmed/kqp179 · Cited by 1,874
When to use agreement versus reliability measures
de Vet, H. C. W., Terwee, C. B., Knol, D. L., & Bouter, L. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.10.015 · Cited by 1,786
A methodological framework for assessing health indices
Kirshner, B., & Guyatt, G. (1985) · Journal of Chronic Diseases · doi:10.1016/0021-9681(85)90005-0 · Cited by 1,538
Epidemiology: Beyond the Basics
Szklo, M., & Nieto, F. J. (1999) · Cited by 1,441
Epidemiology: Beyond the Basics
Szklo, M. (2001) · American Journal of Epidemiology · doi:10.1093/aje/153.8.821 · Cited by 1,310
Measurement reliability and agreement in psychiatry
Shrout, P. E. (1998) · Statistical Methods in Medical Research · doi:10.1177/096228029800700306 · Cited by 1,216
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124 · cited 2×
“However, there is some overlap of the present guidelines with the Standards for Reporting of Diagnostic Accuracy (STARD) (Bossuyt et al., 2003) and with the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, National Council on Measuremen…”
The Delphi technique in health sciences education research
de Villiers, M. R., de Villiers, P. J. T., & Kent, A. P. (2005) · Medical Teacher · doi:10.1080/13611260500069947 · Cited by 1,006
Sample size requirements for estimating intraclass correlations with desired precision
Bonett, D. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1108 · Cited by 987
Misinterpretation and misuse of the kappa statistic
MACLURE, M., & WILLETT, W. C. (1987) · American Journal of Epidemiology · doi:10.1093/aje/126.2.161 · Cited by 808
A critical discussion of intraclass correlation coefficients
Müller, R., & Büttner, P. (1994) · Statistics in Medicine · doi:10.1002/sim.4780132310 · Cited by 714
Sample size requirements for reliability studies
Donner, A., & Eliasziw, M. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060404 · Cited by 699
Self-citationBland, J. M., & Altman, D. G. (1990) · Computers in Biology and Medicine · doi:10.1016/0010-4825(90)90013-f · Cited by 691
Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature
Bot, S. D. M., Terwee, C. B., van der Windt, D. A. W. M., Bouter, L. M., Dekker, J., & de Vet, H. C. W. (2004) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2003.007724 · Cited by 541
Kappa coefficients in medical research
Chmura Kraemer, H., Periyakoil, V. S., & Noda, A. (2002) · Statistics in Medicine · doi:10.1002/sim.1180 · Cited by 527
Wickström (2000) · doi:10.5271/sjweh.555 · Cited by 526
Statistical evaluation of agreement between two methods for measuring a quantitative variable
Lee, J., Koh, D., & Ong, C. N. (1989) · Computers in Biology and Medicine · doi:10.1016/0010-4825(89)90036-x · Cited by 473
Sample size requirements for the design of reliability study: review and new results
Shoukri, M. M., Asyali, M. H., & Donner, A. (2004) · Statistical Methods in Medical Research · doi:10.1191/0962280204sm365ra · Cited by 459
Self-citationStratford, P. W., & Goldsmith, C. H. (1997) · Physical Therapy · doi:10.1093/ptj/77.7.745 · Cited by 455
Vacha-Haase, T. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058001002 · Cited by 449
Assessing intrarater, interrater, and test-retest reliability of continuous measurements
Rousson, V., Gasser, T., & Seifert, B. (2002) · Statistics in Medicine · doi:10.1002/sim.1253 · Cited by 415
Reliability of the Barthel Index when used with older people
Sainsbury, A., Seebass, G., Bansal, A., & Young, J. B. (2005) · Age and Ageing · doi:10.1093/ageing/afi063 · Cited by 374
Cicchetti, D. V. (2001) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.23.5.695.1249 · Cited by 360
Medical Statistics: A Guide to Data Analysis and Critical Appraisal
Peat, J. K., & Barton, B. (2005) · Cited by 339
Ramifications of a Population Model for κ as a Coefficient of Reliability
Kraemer, H. C. (1979) · Psychometrika · doi:10.1007/bf02296208 · Cited by 293
Donner, A., & Eliasziw, M. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111109 · Cited by 266
Self-citationGiraudeau, B., & Mary, J. Y. (2001) · Statistics in Medicine · doi:10.1002/sim.935 · Cited by 255
A Concept for the Validation of Fracture Classifications
Audigé, L., Bhandari, M., Hanson, B., & Kellam, J. F. (2005) · Journal of Orthopaedic Trauma · doi:10.1097/01.bot.0000155310.04886.37 · Cited by 245
Self-citationGoal attainment scaling in paediatric rehabilitation: a critical review of the literature
Steenbeek, D., Ketelaar, M., Galama, K., & Gorter, J. W. (2007) · Developmental Medicine & Child Neurology · doi:10.1111/j.1469-8749.2007.00550.x · Cited by 242
Cicchetti, D., Bronen, R., Spencer, S., Haut, S., Berg, A., Oliver, P., & Tyrer, P. (2006) · Journal of Nervous & Mental Disease · doi:10.1097/01.nmd.0000230392.83607.c5 · Cited by 241
Sample-size calculations for Cohen's kappa.
Cantor, A. B. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.150 · Cited by 240 · cited 1×
“literature (Altaye et al., 2001; Cantor, 1996; Cicchetti, 1999; Donner and Eliasziw, 1987, 1992; Giraudeau and Mary, 2001; Shoukri et al., 2004; Walter et al., 1998).”
Medical Statistics: A Guide to Data Analysis and Critical Appraisal
Lewsey, J. (2006) · The Annals of The Royal College of Surgeons of England · doi:10.1308/rcsann.2006.88.6.603 · Cited by 223
Hestbæk, L., & Leboeuf‐Yde, C. (2000) · Journal of Manipulative and Physiological Therapeutics · doi:10.1067/mmt.2000.106097 · Cited by 195
Use of consensus development to establish national research priorities in critical care
Vella, K. (2000) · BMJ · doi:10.1136/bmj.320.7240.976 · Cited by 194
Manual Examination of the Spine: A Systematic Critical Literature Review of Reproducibility
Stochkendahl, M. J., Christensen, H. W., Hartvigsen, J., Vach, W., Haas, M., Hestbaek, L., Adams, A., & Bronfort, G. (2006) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2006.06.011 · Cited by 188
Audigé, L., Bhandari, M., & Kellam, J. (2004) · Acta Orthopaedica Scandinavica · doi:10.1080/00016470412331294445 · Cited by 186
Self-citationThe dependence of Cohen's kappa on the prevalence does not matter
Vach, W. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.02.021 · Cited by 182
Medical Statistics: A Guide to Data Analysis and Critical Appraisal
Peat, J., & Barton, B. (2005) · doi:10.1002/9780470755945 · Cited by 181
Measures of interobserver agreement: Calculation formulas and distribution effects
House, A. E., House, B. J., & Campbell, M. B. (1981) · Journal of Behavioral Assessment · doi:10.1007/bf01321350 · Cited by 180
A Reappraisal of Reliability and Validity Studies in Stroke
D'Olhaberriague, L., Litvan, I., Mitsias, P., & Mansbach, H. H. (1996) · Stroke · doi:10.1161/01.str.27.12.2331 · Cited by 173
Inter‐rater reliability of the EPUAP pressure ulcer classification system using photographs
Defloor, T., & Schoonhoven, L. (2004) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2004.00974.x · Cited by 172
Slongo, T. F., Audigé, L., Schlickewei, W., Clavert, J.-M., & Hunter, J. B. (2005) · Journal of Pediatric Orthopaedics · doi:10.1097/01.bpo.0000187989.64021.ml · Cited by 171
Self-citationIntertester Reliability and Diagnostic Validity of the Cervical Flexion-Rotation Test
Hall, T. M., Robinson, K. W., Fujinawa, O., Akasaka, K., & Pyne, E. A. (2008) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2008.03.012 · Cited by 164
Reliability of work-related assessments
Innes, E., & Straker, L. (1999) · WORK: A Journal of Prevention, Assessment & Rehabilitation · doi:10.3233/wor-1999-00051 · Cited by 160
How reliable are assessments of clinical teaching? A review of the published instruments
Beckman, T. J., Ghosh, A. K., Cook, D. A., Erwin, P. J., & Mandrekar, J. N. (2004) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2004.40066.x · Cited by 160
Statistical Evaluation of Measurement Errors: Design and Analysis of Reliability Studies
Dunn, G. (2004) · Cited by 147
Bours, G. J. J. W., Halfens, R. J. G., Lubbers, M. J., & Haalboom, J. R. (1999) · PubMed · Cited by 134
Measures of interobserver agreement
Shoukri, M. M. (2004) · Thammasat University Digital Collections · Cited by 133
Self-citationMeasures of interobserver agreement: Calculation formulas and distribution effects
House, A. E. (1982) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/bf01321270 · Cited by 128
A comparison of formal consensus methods used for developing clinical guidelines
Hutchings, A., Raine, R., Sanderson, C., & Black, N. (2006) · Journal of Health Services Research & Policy · doi:10.1258/135581906778476553 · Cited by 125
Gouttebarge, V., Wind, H., Kuijer, P., & Frings‐Dresen, M. H. W. (2004) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-004-0549-7 · Cited by 123
Measurement of reliability for categorical data in medical research
Kraemer, H. C. (1992) · Statistical Methods in Medical Research · doi:10.1177/096228029200100204 · Cited by 114
Perkins, D. O., Wyatt, R. J., & Bartko, J. J. (2000) · Biological Psychiatry · doi:10.1016/s0006-3223(00)00837-4 · Cited by 111
Sun, Y., Stürmer, T., Gunther, K. P., & Brenner, H. (1997) · Clinical Rheumatology · doi:10.1007/bf02247849 · Cited by 108
Clinimetrics vs. psychometrics: an unnecessary distinction
Streiner, D. L. (2003) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.08.011 · Cited by 107
Self-citationWhy we need large, simple studies of the clinical examination: the problem and a proposed solution
McAlister, F. A., Straus, S. E., & Sackett, D. L. (1999) · The Lancet · doi:10.1016/s0140-6736(99)01174-5 · Cited by 99
Category Distinguishability and Observer Agreement
Darroch, J. N., & McCloud, P. (1986) · Australian Journal of Statistics · doi:10.1111/j.1467-842x.1986.tb00709.x · Cited by 99
Effective number of subjects and number of raters for inter‐rater reliability studies
Saito, Y., Sozu, T., Hamada, C., & Yoshimura, I. (2005) · Statistics in Medicine · doi:10.1002/sim.2294 · Cited by 96
The statistical analysis of kappa statistics in multiple samples
Donner, A. P., & Klar, N. S. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(96)00057-1 · Cited by 94
Self-citationZegers, M., de Bruijne, M. C., Wagner, C., Groenewegen, P., van der Wal, G., & de Vet, H. C. W. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.03.004 · Cited by 87
Richardson, F. M. (1972) · Medical Care · doi:10.1097/00005650-197201000-00004 · Cited by 84
Subepidermal moisture differentiates erythema and stage I pressure ulcers in nursing home residents
Bates‐Jensen, B. M., McCreath, H. E., Pongquan, V., & Apeles, N. C. R. (2008) · Wound Repair and Regeneration · doi:10.1111/j.1524-475x.2008.00359.x · Cited by 80
Reliability Testing of the National Database of Nursing Quality Indicators Pressure Ulcer Indicator
Hart, S., Bergquist, S., Gajewski, B. J., & Dunton, N. E. (2006) · Journal of Nursing Care Quality · doi:10.1097/00001786-200607000-00011 · Cited by 78
Self-citationA Nonparametric Measure of Intraclass Correlation
ROTHERY, P. (1979) · Biometrika · doi:10.1093/biomet/66.3.629 · Cited by 77
Ratanawongsa, N., Thomas, P. A., Marinopoulos, S. S., Dorman, T., Wilson, L. M., Ashar, B. H., Magaziner, J. L., Miller, R. G., Prokopowicz, G. P., Qayyum, R., et al. (2008) · Academic Medicine · doi:10.1097/acm.0b013e3181637925 · Cited by 77
Measures of Interobserver Agreement
Nagaraja, H. N. (2004) · Technometrics · doi:10.1198/tech.2004.s205 · Cited by 74
Mok, J. M., Berven, S., Diab, M., Hackbarth, M., Hu, S. S., & Deviren, V. (2008) · Spine · doi:10.1097/brs.0b013e318166aa8d · Cited by 72
The Kappa Coefficient and the Prevalence of a Diagnosis
Gjørup, T. (1988) · Methods of Information in Medicine · doi:10.1055/s-0038-1635539 · Cited by 71
Pressure ulcer prevalence, incidence and associated risk factors in the community.
Oot-Giromini, B. (1993) · PubMed · Cited by 70
Colle, F., Rannou, F., Revel, M., Fermanian, J., & Poiraudeau, S. (2002) · Archives of Physical Medicine and Rehabilitation · doi:10.1053/apmr.2002.35657 · Cited by 68 · cited 2×
“When continuous measurements are split into distinct categories (see item 2), it is recommended that results be calculated for the continuous measurement as well, because the transformation of continuous values into categories may cause difficulties in interpretation and lead to a reduction in statistical power [32,58…”
Donner, A., & Eliasziw, M. (1994) · Biometrics · doi:10.2307/2533400 · Cited by 66
Self-citationAn interrater reliability study of the Braden scale in two nursing homes
Kottner, J., & Dassen, T. W. N. (2008) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2008.02.007 · Cited by 65
Self-citationEPUAP statement on prevalence and incidence monitoring of pressure ulcer occurrence
Defloor, T., Clark, M., Witherow, A., Colin, D., Lindholm, C., Schoonhoven, L., & Moore, Z. E. H. (2005) · Journal of Tissue Viability · doi:10.1016/s0965-206x(05)53004-3 · Cited by 65
A systematic review of interrater reliability of pressure ulcer classification systems
Kottner, J., Raeder, K., Halfens, R. J. G., & Dassen, T. W. N. (2009) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2008.02569.x · Cited by 62
Self-citationA comparison of methods for calculating a stratified kappa
Barlow, W. E., Lai, M., & Azen, S. P. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100913 · Cited by 61
Swingler, G. H. (2001) · BMC Medical Imaging · doi:10.1186/1471-2342-1-1 · Cited by 60
Inter- and intrarater reliability of the Waterlow pressure sore risk scale: A systematic review
Kottner, J., Dassen, T. W. N., & Tannen, A. (2008) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2008.09.010 · Cited by 57
Self-citationUsing Behavioral Science Strategies for Defining the State-of-the-Art
Glaser, E. M. (1980) · The Journal of Applied Behavioral Science · doi:10.1177/002188638001600107 · Cited by 54
Sensitivity and Specificity of the Braden Scale in the Cardiac Surgical Population
Lewicki, L. J., Mion, L. C., & Secic, M. (2000) · Journal of Wound Ostomy and Continence Nursing · doi:10.1097/00152192-200001000-00016 · Cited by 54
Interpreting interrater reliability coefficients of the Braden scale: A discussion paper
Kottner, J., & Dassen, T. W. N. (2007) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2007.08.001 · Cited by 53
Self-citationBååth, C., Hall‐Lord, M., Idvall, E., Wiberg-Hedman, K., & Wilde‐Larsson, B. (2008) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2007.02131.x · Cited by 52
Interrater Reliability in Clinical Trials of Depressive Disorders
Mulsant, B. H., Kastango, K. B., Rosen, J., Stone, R. A., Mazumdar, S., & Pollock, B. G. (2002) · American Journal of Psychiatry · doi:10.1176/appi.ajp.159.9.1598 · Cited by 52
Rater Training in Multicenter Clinical Trials: Issues and Recommendations
Kobak, K. A., Engelhardt, N., Williams, J. B. W., & Lipsitz, J. D. (2004) · Journal of Clinical Psychopharmacology · doi:10.1097/01.jcp.0000116651.91923.54 · Cited by 51
Brorson, S., & Hróbjartsson, A. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.04.014 · Cited by 50
Self-citationCocchetti, D. V. (1999) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.21.4.567.886 · Cited by 43
A New Approach to Rater Training and Certification in a Multicenter Clinical Trial
Kobak, K. A., Lipsitz, J. D., Williams, J. B. W., Engelhardt, N., & Bellew, K. M. (2005) · Journal of Clinical Psychopharmacology · doi:10.1097/01.jcp.0000177666.35016.a0 · Cited by 42
Should an Allen Test Be Performed Before Radial Artery Cannulation?
Barone, J. E., & Madlinger, R. V. (2006) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/01.ta.0000229815.43871.59 · Cited by 42
A general goodness‐of‐fit approach for inference procedures concerning the kappa statistic
Altaye, M., DONNER, A. P., & Eliasziw, M. (2001) · Statistics in Medicine · doi:10.1002/sim.911 · Cited by 42
Self-citationGajewski, B. J., Hart, S., Bergquist‐Beringer, S., & Dunton, N. E. (2007) · Statistics in Medicine · doi:10.1002/sim.2877 · Cited by 41
Self-citationExamining the validity of pressure ulcer risk assessment scales: a replication study
Gould, D. J., Goldstone, L., Kelly, D. M., & Gammon, J. (2003) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2003.10.005 · Cited by 40
Inter-observer variation in the assessment of skin ulceration
Buntinx, F., Beckers, H., Briers, A., de Keyser, G., Flour, M., Nissen, G., Raskin, T., & de Vet, H. C. W. (1996) · Journal of Wound Care · doi:10.12968/jowc.1996.5.4.166 · Cited by 37
A matrix of kappa-type coefficients to assess the reliability of nominal scales
Roberts, C., & McNamee, R. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980228)17:4<471::aid-sim745>3.0.co;2-n · Cited by 36
Self-citationKadam, U. T., Jordan, K. P., & Croft, P. R. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.016 · Cited by 35
Vikström, A. C., Skånér, Y., Strender, L., & Nilsson, G. H. (2007) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-7-9 · Cited by 34
Kobak, K. A., Williams, J. B. W., & Engelhardt, N. (2007) · Psychiatry Research · doi:10.1016/j.psychres.2007.06.025 · Cited by 31
Nanda, R., Gupta, S., Kanapathipillai, P., Liow, R., & Rangan, A. (2008) · European Journal of Orthopaedic Surgery & Traumatology · doi:10.1007/s00590-008-0341-6 · Cited by 31
Thomsen, N., Olsen, L. H., & Nielsen, S. T. (2002) · Journal of Orthopaedic Science · doi:10.1007/s007760200028 · Cited by 31
Is Real-Time Ultrasonic Bladder Volume Estimation Reliable and Valid? A Systematic Overview
Chika R. Nwosu, K. S. K. (1998) · Scandinavian Journal of Urology and Nephrology · doi:10.1080/003655998750015278 · Cited by 28
Representation of Ophthalmology Concepts by Electronic Systems
Hwang, J. C., Yu, A. C., Casper, D. S., Starren, J. B., Cimino, J. J., & Chiang, M. F. (2006) · Ophthalmology · doi:10.1016/j.ophtha.2006.01.017 · Cited by 25
Assessing the reliability of ordered categorical scales using kappa-type statistics
Roberts, C., & McNamee, R. (2005) · Statistical Methods in Medical Research · doi:10.1191/0962280205sm413oa · Cited by 25
Self-citationInterrater Reliability Decline Under Covert Assessment
Topf, M. (1988) · Nursing Research · doi:10.1097/00006199-198801000-00010 · Cited by 24
Signal-to-noise ratio (SNR) as a measure of reproducibility: design, estimation, and application
Elkum, N. B., & Shoukri, M. M. (2008) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-008-0030-2 · Cited by 18
Self-citationCharter, R. A. (2003) · Psychological Reports · doi:10.2466/pr0.2003.93.3.643 · Cited by 17
Inter-Rater Reliability of Delirium Rating Scales
Bhat, R. S., & Rockwood, K. J. (2005) · Neuroepidemiology · doi:10.1159/000085440 · Cited by 16
The case for comprehensive quality indicator reliability assessment
Scinto, J. D., Galusha, D. H., Krumholz, H. M., & Meehan, T. P. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00381-x · Cited by 14
Whitfield, J. W. (1949) · Biometrika · doi:10.1093/biomet/36.3-4.463 · Cited by 11
Charter, R. A. (2003) · Psychological Reports · doi:10.2466/pr0.93.7.643-647 · Cited by 2
Walter (1998) · doi:10.1002/(sici)1097-0258(19980115)17:1<101::aid-sim727>3.0.co;2-e
Self-citation(1994)
Cited by
SANRA—a scale for the quality assessment of narrative review articles
Baethge, C., Goldbeck-Wood, S., & Mertens, S. (2019) · Research Integrity and Peer Review · doi:10.1186/s41073-019-0064-8 · Cited by 3,469
A catalogue of reporting guidelines for health research
Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. (2010) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2009.02234.x · Cited by 1,047
Interrater agreement and interrater reliability: Key concepts, approaches, and applications
Gisev, N., Bell, J. S., & Chen, T. F. (2013) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2012.04.004 · Cited by 924
Gagnier, J. J., Lai, J., Mokkink, L. B., & Terwee, C. B. (2021) · Quality of Life Research · doi:10.1007/s11136-021-02822-4 · Cited by 881
Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., Schwartz, C., Revicki, D. A., Moinpour, C. M., McLeod, L. D., et al. (2013) · Quality of Life Research · doi:10.1007/s11136-012-0344-y · Cited by 845
Metadata
- Authors
- Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2011-01
- Volume
- 64
- Issue
- 1
- Pages
- 96-106
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2069378996
- Cited by
- 2,174
- References
- 126
- FWCI
- 36.39
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.