The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Kurtzke scales revisited: the application of psychometric methods to clinical intuition
Abstract
When developing his disability scales for multiple sclerosis, Kurtzke demonstrated perception and insight. However, 45 years later, the evaluation of his clinically derived scales remains limited, particularly for more disabled patients. Indeed, many of Kurtzke's assumptions underpinning the development of the Expanded Disability Status Scale (EDSS) and Functional Systems (FS) are untested. This study aims to build on previous work and provide a more detailed examination using psychometric methods of the EDSS and FS. There are three study objectives: (i) to examine comprehensively the psychometric properties of the EDSS in more disabled people with multiple sclerosis undergoing in-patient rehabilitation; (ii) to examine the reliability of the FS and test Kurtzke's assumptions that they measure different aspects of the neurological examination and measure different constructs from that measured by the EDSS; and (iii) to examine whether the FS can be summed to generate a summary score. The EDSS was examined for its acceptability (score distributions), reliability (inter- and intra-rater reproducibility, standard error of measurement), validity (convergent and discriminant validity, measurement precision, discrimination between individuals) and responsiveness (effect size). The FS were examined for their reliability (inter- and intra-rater reproducibility), intercorrelations, correlations with the EDSS and the extent to which they satisfy Likert's criteria as a summed rating scale. In this more disabled sample of people with multiple sclerosis, the EDSS is an acceptable measure but demonstrates limited variability. Inter-rater reproducibility (intraclass correlation coefficient; ICC = 0.78) is adequate for group comparison studies, but intra-rater reproducibility is variable (ICC = 0.62-0.94). Convergent and discriminant validity for the EDSS is supported, but its measurement precision relative to the Functional Independence Measure is limited (56%). Also, the EDSS has a limited ability to distinguish between individuals in terms of their disability and its responsiveness is poor (effect size = 0.10). Results indicate that the FS measure constructs distinct from each other (intercorrelations = -0.23 to +0. 52) and from the EDSS (correlations = -0.10 to +0.59). Intra-rater, but not inter-rater reproducibility is adequate for group comparison studies. The FS do not satisfy criteria as an eight-, seven- or six-item summed rating scale. Despite being based on sound clinical intuition, the lack of psychometric input into the development of the EDSS and FS has limited their usefulness as evaluative outcome measures in multiple sclerosis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
(1992) · PubMed · Cited by 33,716
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
EuroQol: a new facility for the measurement of health-related quality of life
(1990) · Health Policy · doi:10.1016/0168-8510(90)90421-9 · Cited by 15,736
Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS)
Kurtzke, J. F. (1983) · Neurology · doi:10.1212/wnl.33.11.1444 · Cited by 15,011
Functional evaluation : Barthel index
Mahonery, F. I. (1965) · Medical Entomology and Zoology · Cited by 12,762
SF-36 health survey: Manual and interpretation guide
Ware, J. (2003) · Medical Entomology and Zoology · Cited by 11,424
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
New diagnostic criteria for multiple sclerosis: Guidelines for research protocols
Poser, C. M., Paty, D. W., Scheinberg, L., McDonald, W. I., Davis, F. A., Ebers, G. C., Johnson, K. P., Sibley, W. A., Silberberg, D. H., & Tourtellotte, W. W. (1983) · Annals of Neurology · doi:10.1002/ana.410130302 · Cited by 7,384
MCHORNEY, C. A., JOHNE, W., & ANASTASIAE, R. (1993) · Medical Care · doi:10.1097/00005650-199303000-00006 · Cited by 6,681
A scaled version of the General Health Questionnaire
Goldberg, D. P., & Hillier, V. F. (1979) · Psychological Medicine · doi:10.1017/s0033291700021644 · Cited by 5,459
Essentials of Psychological Testing
Rosner, B., & Cronbach, L. J. (1960) · The American Journal of Psychology · doi:10.2307/1419921 · Cited by 4,513
McHorney, C. A., Ware, J. E., Rachel Lu, J. F., & Sherbourne, C. D. (1994) · Medical Care · doi:10.1097/00005650-199401000-00004 · Cited by 4,458
(1994) · Medical Entomology and Zoology · Cited by 4,156
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,883
The Barthel ADL Index: A reliability study
Collin, C., Wade, D. T., Davies, S., & Horne, V. (1988) · International Disability Studies · doi:10.3109/09638288809164103 · Cited by 2,734
Measuring Functioning and Well-Being: The Medical Outcomes Study Approach
Stewart, A., & Ware, J. (1992) · doi:10.7249/cb361 · Cited by 2,689
Cerebral Vascular Accidents in Patients over the Age of 60: II. Prognosis
Rankin, J. (1957) · Scottish Medical Journal · doi:10.1177/003693305700200504 · Cited by 2,506
Intramuscular interferon beta‐1a for disease progression in relapsing multiple sclerosis
Jacobs, L. D., Cookfair, D. L., Rudick, R. A., Herndon, R. M., Richert, J. R., Salazar, A. M., Fischer, J. S., Goodkin, D. E., Granger, C. V., Simon, J. H., et al. (1996) · Annals of Neurology · doi:10.1002/ana.410390304 · Cited by 2,450
Effect Sizes for Interpreting Changes in Health Status
Kazis, L. E., Anderson, J. J., & Meenan, R. F. (1989) · Medical Care · doi:10.1097/00005650-198903001-00015 · Cited by 2,401
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,299
Individual-patient monitoring in clinical practice: are available health status surveys adequate?
McHorney, C. A., & Tarlov, A. R. (1995) · Quality of Life Research · doi:10.1007/bf01593882 · Cited by 2,206
Summated Rating Scale Construction
Spector, P. (1992) · Sage eBooks · doi:10.4135/9781412986038 · Cited by 2,158
Measuring change over time: Assessing the usefulness of evaluative instruments
Guyatt, G., Walter, S., & Norman, G. (1987) · Journal of Chronic Diseases · doi:10.1016/0021-9681(87)90069-5 · Cited by 1,880
Evaluating patient-based outcome measures for use in clinical trials.
Fitzpatrick, Davey, Buxton, & Jones (1998) · Health Technology Assessment · doi:10.3310/hta2140 · Cited by 1,715
PRELIMINARY TRIAL OF CARISOPRODOL IN MULTIPLE SCLEROSIS.
Ashworth, B. (1964) · PubMed · Cited by 1,475
Thurstone, L. L. (1928) · American Journal of Sociology · doi:10.1086/214483 · Cited by 1,464
Development of a multiple sclerosis functional composite as a clinical trial outcome measure
Cutter, G. R. (1999) · Brain · doi:10.1093/brain/122.5.871 · Cited by 1,190
A health-related quality of life measure for multiple sclerosis
Vickrey, B. G., Hays, R. D., Harooni, R., Myers, L. W., & Ellison, G. W. (1995) · Quality of Life Research · doi:10.1007/bf02260859 · Cited by 1,132
Measurement in Neurological Rehabilitation
Cole, M. (1993) · Neurology · doi:10.1212/wnl.43.2.459-b · Cited by 1,011
Advances in functional assessment for medical rehabilitation
Granger, C. V., Hamilton, B. B., Keith, R. A., Zielezny, M., & Sherwin, F. S. (1986) · Topics in Geriatric Rehabilitation · doi:10.1097/00013614-198604000-00007 · Cited by 778
Conceptualization and Measurement of Health for Adults in the Health Insurance Study
Stewart, A., Ware, J. E., Brook, R. H., & Davies, A. R. (1978) · Cited by 641
Liang, M. H., Larson, M. G., Cullen, K. E., & Schwartz, J. A. (1985) · Arthritis & Rheumatism · doi:10.1002/art.1780280513 · Cited by 588
Norman, G. R., Stratford, P., & Regehr, G. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00097-8 · Cited by 524
McIver, J., & Carmines, E. (1981) · Sage eBooks · doi:10.4135/9781412986441 · Cited by 500
Validation of the Functional Assessment of Multiple Sclerosis quality of life instrument
Cella, D. F., Dineen, K., Arnason, B., Reder, A., Webster, K. A., Karabatsos, G., Chang, C., Lloyd, S., Mo, F., Stewart, J., et al. (1996) · Neurology · doi:10.1212/wnl.47.1.129 · Cited by 413
Functional assessment scales: a study of persons with multiple sclerosis.
Cv, G., Ac, C., Bb, H., Fiedler, R., & Hens, M. M. (1990) · PubMed · Cited by 385
On the evaluation of disability in multiple sclerosis
Kurtzke, J. F. (1961) · Neurology · doi:10.1212/wnl.11.8.686 · Cited by 330
Noseworthy, J. H., Vandervoort, M. K., Wong, C. J., & Ebers, G. C. (1990) · Neurology · doi:10.1212/wnl.40.6.971 · Cited by 328
Further notes on disability evaluation in multiple sclerosis, with scale modifications
Kurtzke, J. F. (1965) · Neurology · doi:10.1212/wnl.15.7.654 · Cited by 308
A New Scale for Evaluating Disability in Multiple Sclerosis
Kurtzke, J. F. (1955) · Neurology · doi:10.1212/wnl.5.8.580 · Cited by 297
MAP-R for windows: multitrait / multi-item analysis program - revised user's guide.
Ware, J., Harris, W. P., Gandek, B., Rogers, B., & Reese, P. (1997) · Cited by 269
Overview of adult health measures fielded in Rand's health insurance study.
Brook, R. H., Ware, J. E., Davies-Avery, A., Stewart, A. L., Donald, C. A., Rogers, W. H., Williams, K. N., & Johnston, S. A. (1979) · PubMed · Cited by 258
The psychometric properties of clinical rating scales used in multiple sclerosis
Sharrack, B., Hughes, R. A. C., Soudain, S., & Dunn, G. (1999) · Brain · doi:10.1093/brain/122.1.141 · Cited by 258
Outcomes assessment in multiple sclerosis clinical trials: a critical analysis
Whitaker, J., McFarland, H., Rudge, P., & Reingold, S. (1995) · Multiple Sclerosis Journal · doi:10.1177/135245859500100107 · Cited by 236
The impact of inpatient rehabilitation on progressive multiple sclerosis
Freeman, J. A., Langdon, D. W., Hobart, J. C., & Thompson, A. J. (1997) · Annals of Neurology · doi:10.1002/ana.410420216 · Cited by 234
Self-citationMeasuring Components of Childrenʼs Health Status
Eisen, M., Ware, J. E., Donald, C. A., & Brook, R. H. (1979) · Medical Care · doi:10.1097/00005650-197909000-00003 · Cited by 226
Goodkin, D. E., Cookfair, D., Wende, K., Bourdette, D., Pullicino, P., Scherokman, B., & Whitham, R. (1992) · Neurology · doi:10.1212/wnl.42.4.859 · Cited by 212
NEUROLOGIC IMPAIRMENT IN MULTIPLE SCLEROSIS AND THE DISABILITY STATUS SCALE
Kurtzke, J. F. (1970) · Acta Neurologica Scandinavica · doi:10.1111/j.1600-0404.1970.tb05808.x · Cited by 168
Amato, M. P., Fratiglioni, L., Groppi, C., Siracusa, G., & Amaducci, L. A. (1988) · Archives of Neurology · doi:10.1001/archneur.1988.00520310052017 · Cited by 162
Scales for rating impairment in multiple sclerosis: A critique
Willoughby, E. W., & Paty, D. W. (1988) · Neurology · doi:10.1212/wnl.38.11.1793 · Cited by 158
A Method for Correcting Item-Total Correlations for the Effect of Relevant Item Inclusion
Howard, K. I., & Forehand, G. A. (1962) · Educational and Psychological Measurement · doi:10.1177/001316446202200407 · Cited by 148
Development Standards for Health Measures
McDowell, I., & Jenkinson, C. (1996) · Journal of Health Services Research & Policy · doi:10.1177/135581969600100410 · Cited by 121
An Assessment of Disability Rating Scales Used in Multiple Sclerosis
Francis, D. A., Bain, P. G., Swan, A. V., & Hughes, R. A. (1991) · Archives of Neurology · doi:10.1001/archneur.1991.00530150067020 · Cited by 109
Clinical scales for multiple sclerosis
Sharrack, B., & Hughes, R. A. (1996) · Journal of the Neurological Sciences · doi:10.1016/0022-510x(95)00261-y · Cited by 107
Introduction to Statistics for Psychology and Education
Clayton, K. N. (1984) · Medical Entomology and Zoology · Cited by 71
Responsiveness of the Rehabilitation Activities Profile and the Barthel Index
van Bennekom, C. A. M., Jelles, F., Lankhorst, G. J., & Bouter, L. M. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00559-5 · Cited by 66
SF-20 Score and Item Distributions in a Human Immunodeficiency Virus-Seropositive Sample
Holmes, W. C., Bix, B. C., & Shea, J. A. (1996) · Medical Care · doi:10.1097/00005650-199606000-00006 · Cited by 65
Meta-analysis of the placebo-treated groups in clinical trials of progressive MS
Weinshenker, B. G., Issa, M., & Baskerville, J. C. (1996) · Neurology · doi:10.1212/wnl.46.6.1613 · Cited by 64
TRANSITION QUESTIONS TO ASSESS OUTCOMES IN RHEUMATOID ARTHRITIS
Fitzpatrick, R. M., Ziébland, S., JENKISON, C., Mowat, A. N., & Mowat, A. N. (1993) · Rheumatology · doi:10.1093/rheumatology/32.9.807 · Cited by 55
Dissent How should health status measures be assessed? Cautionary notes on procrustean frameworks
Williams, J. I., & Naylor, C. D. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90195-s · Cited by 51
Assessing health-related quality of life in HIV disease: key measurement issues
Shumaker, S. A., Ellis, S. D., & McNaughton, L. R. (1997) · Quality of Life Research · doi:10.1023/a:1018499813912 · Cited by 49
Multiple sclerosis : advances in clinical trial design, treatment, and future perspectives
Goodkin, D. E., & Rudick, R. A. (1996) · Springer eBooks · Cited by 27
LaRocca, N. G., Ritvo, P. G., Miller, D. M., Fischer, J. S., Andrews, H., & Paty, D. W. (1996) · Multiple Sclerosis · doi:10.1007/978-1-4471-1271-6_7 · Cited by 21
ISONIAZID IN TREATMENT OF MULTIPLE SCLEROSIS
Kurtzke, J. F. (1957) · JAMA · doi:10.1001/jama.1957.02970380014005 · Cited by 16
Cited by
The Multiple Sclerosis Impact Scale (MSIS-29): A new patient-based outcome measure
Hobart, J. (2001) · Brain · doi:10.1093/brain/124.5.962 · Cited by 1,120
Self-citationMeasuring the impact of MS on walking ability: The 12-Item MS Walking Scale (MSWS-12)
Hobart, J. C., Riazi, A., Lamping, D. L., Fitzpatrick, R., & Thompson, A. J. (2003) · Neurology · doi:10.1212/wnl.60.1.31 · Cited by 821
Self-citationLearmonth, Y. C., Motl, R. W., Sandroff, B. M., Pula, J. H., & Cadavid, D. (2013) · BMC Neurology · doi:10.1186/1471-2377-13-37 · Cited by 773
Meyer-Moock, S., Feng, Y.-S., Maeurer, M., Dippel, F.-W., & Kohlmann, T. (2014) · BMC Neurology · doi:10.1186/1471-2377-14-58 · Cited by 639
Hobart, J., & Cano, S. (2009) · Health Technology Assessment · doi:10.3310/hta13120 · Cited by 621
Self-citation
Metadata
- Authors
- Hobart, J.
- Venue
- Brain
- Publisher
- Oxford University Press (OUP)
- Published
- 2000-05-01
- Volume
- 123
- Issue
- 5
- Pages
- 1027-1040
- ISSN
- 0006-8950 (ISSN-L)
- 1460-2156
- OpenAlex
- W1967990636
- Cited by
- 328
- References
- 67
- FWCI
- 7.27
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.