Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods
Open accessThe DOI matches its Crossref record; 241 of the 241 references it cites were found in a registry or an index.
Abstract
OBJECTIVES: In this monograph we examine the added value of new psychometric methods (Rasch measurement and Item Response Theory) over traditional psychometric approaches by comparing and contrasting their psychometric evaluations of existing sets of rating scale data. We have concentrated on Rasch measurement rather than Item Response Theory because we believe that it is the more advantageous method for health measurement from a conceptual, theoretical and practical perspective. Our intention is to provide an authoritative document that describes the principles of Rasch measurement and the practice of Rasch analysis in a clear, detailed, non-technical form that is accurate and accessible to clinicians and researchers in health measurement. REVIEW METHODS: A comparison was undertaken of traditional and new psychometric methods in five large sets of rating scale data: (1) evaluation of the Rivermead Mobility Index (RMI) in data from 666 participants in the Cannabis in Multiple Sclerosis (CAMS) study; (2) evaluation of the Multiple Sclerosis Impact Scale (MSIS-29) in data from 1725 people with multiple sclerosis; (3) evaluation of test-retest reliability of MSIS-29 in data from 150 people with multiple sclerosis; (4) examination of the use of Rasch analysis to equate scales purporting to measure the same health construct in 585 people with multiple sclerosis; and (5) comparison of relative responsiveness of the Barthel Index and Functional Independence Measure in data from 1400 people undergoing neurorehabilitation. RESULTS: Both Rasch measurement and Item Response Theory are conceptually and theoretically superior to traditional psychometric methods. Findings from each of the five studies show that Rasch analysis is empirically superior to traditional psychometric methods for evaluating rating scales, developing rating scales, analysing rating scale data, understanding and measuring stability and change, and understanding the health constructs we seek to quantify. CONCLUSIONS: There is considerable added value in using Rasch analysis rather than traditional psychometric methods in health measurement. Future research directions include the need to reproduce our findings in a range of clinical populations, detailed head-to-head comparisons of Rasch analysis and Item Response Theory, and the application of Rasch analysis to clinical practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“To this end, the Rasch measurement model [14] provides a powerful alternative by determining the extent to which observed data confirm with the model [11 -15].”
From Construct validity of a revised version of the Occupational Balance Questionnaire (2020) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“However, CTT approaches to assessing the measurement properties of scales have limitations [11 -13].”
From Construct validity of a revised version of the Occupational Balance Questionnaire (2020) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–241 of 241
Applied Rasch Measurement: A Book of Exemplars
Maclean, R., Watanabe, R., Thaman, K. H., Rajput, J. S., Power, C., Mansheng, Z., Keeves, J. P., Hungi, N., Duncan, W., Curtis, D., et al. (2005) · Springer eBooks · doi:10.1007/1-4020-3076-2 · Cited by 115
Roderick, P., Nicholson, T., Armitage, A., Mehta, R., Mullee, M., Gerard, K., Drey, N., Feest, T., Greenwood, R., Lamping, D., et al. (2005) · Health Technology Assessment · doi:10.3310/hta9240 · Cited by 114
Bridle, C., Palmer, S., Bagnall, A.-M., Darba, J., Duffy, S., Sculpher, M., & Riemsma, R. (2004) · Health Technology Assessment · doi:10.3310/hta8190 · Cited by 114
Garside, R., Stein, K., Castelnuovo, E., Pitt, M., Ashcroft, D., Dimmock, P., & Payne, L. (2005) · Health Technology Assessment · doi:10.3310/hta9290 · Cited by 112
How accurate are lexile text measures?
Stenner, A. J., Burdick, H., Sanford, E. E., & Burdick, D. S. (2006) · PubMed · Cited by 111
Finfer, S. R. (2006) · Journal of the Japan Society of Blood Transfusion · doi:10.3925/jjtc1958.52.19 · Cited by 111
Hummel, S., Paisley, S., Morgan, A., Currie, E., & Brewer, N. (2003) · Health Technology Assessment · doi:10.3310/hta7330 · Cited by 111
Berry, E., Kelly, S., Westwood, M. E., Davies, L. M., Gough, M. J., Bamford, J. M., Meaney, J. F. M., Airey, C. M., Cullingworth, J., Barbieri, M., et al. (2002) · Health Technology Assessment · doi:10.3310/hta6070 · Cited by 111
Sculpher, Petticrew, Kelland, Elliott, Holdright, & Buxton (1998) · Health Technology Assessment · doi:10.3310/hta2100 · Cited by 105
Stein, K., Dalziel, K., Walker, A., McIntyre, L., Jenkins, B., Horne, J., Royle, P., & Round, A. (2002) · Health Technology Assessment · doi:10.3310/hta6310 · Cited by 103
Hoogervorst, E. L. J., Zwemmer, J., Jelles, B., Polman, C. H., & Uitdehaag, B. M. J. (2004) · Multiple Sclerosis Journal · doi:10.1191/1352458504ms1078oa · Cited by 99
Wilson, B., Torrance, N., Mollison, J., Wordsworth, S., Gray, J., Haites, N., Grant, A., Campbell, M., Miedyzbrodzka, Z., Clarke, A., et al. (2005) · Health Technology Assessment · doi:10.3310/hta9030 · Cited by 97
Willis, B., Barton, P., Pearmain, P., Bryan, S., & Hyde, C. (2005) · Health Technology Assessment · doi:10.3310/hta9130 · Cited by 93
Riazi, A., Hobart, J. C., Lamping, D. L., Fitzpatrick, R., & Thompson, A. J. (2003) · Multiple Sclerosis Journal · doi:10.1191/1352458503ms929oa · Cited by 91
Self-citationFludarabine as second-line therapy for B cell chronic lymphocytic leukaemia: a technology assessment
Hyde, C., Wake, B., Bryan, S., Barton, P., Fry-Smith, A., Davenport, C., & Song, F. (2002) · Health Technology Assessment · doi:10.3310/hta6020 · Cited by 90
Elliott, R. A., Payne, K., Moore, J. K., Davies, L. M., Harper, N. J. N., St. Leger, A. S., Moore, E. W., Thoms, G. M. M., Pollard, B. J., McHugh, G. A., et al. (2002) · Health Technology Assessment · doi:10.3310/hta6300 · Cited by 89
DOES THE RASCH MODEL REALLY WORK FOR MULTIPLE CHOICE ITEMS? NOT IF YOU LOOK CLOSELY
Divgi, D. R. (1986) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1986.tb00251.x · Cited by 87
A postal version of the Barthel Index
Gompertz, P., Pound, P., & Ebrahim, S. B. J. (1994) · Clinical Rehabilitation · doi:10.1177/026921559400800308 · Cited by 87
Bagnall, A., Jones, L., Richardson, G., Duffy, S., & Riemsma, R. (2003) · Health Technology Assessment · doi:10.3310/hta7190 · Cited by 86
Symposium: Measurement and its Importance for Philosophy
Campbell, N. R., & Jeffreys, H. (1938) · Aristotelian Society Supplementary Volume · doi:10.1093/aristoteliansupp/17.1.121 · Cited by 81
Jones, L., Hawkins, N., Westwood, M., Wright, K., Richardson, G., & Riemsma, R. (2004) · Health Technology Assessment · doi:10.3310/hta8050 · Cited by 81
McDonagh, Bachmann, Golder, Kleijnen, & ter Riet (2000) · Health Technology Assessment · doi:10.3310/hta4300 · Cited by 79
Autoantibody testing in children with newly diagnosed type 1 diabetes mellitus
Dretzke, J., Cummins, C., Sandercock, J., Fry-Smith, A., Barrett, T., & Burls, A. (2004) · Health Technology Assessment · doi:10.3310/hta8220 · Cited by 77
Andrich, D. (2005) · Springer eBooks · doi:10.1007/1-4020-3076-2_3 · Cited by 73
Stenner, A. J., & Smith, M. (1982) · Perceptual and Motor Skills · doi:10.2466/pms.1982.55.2.415 · Cited by 71
Choosing response categories for summated rating scales.
Spector, P. E. (1976) · Journal of Applied Psychology · doi:10.1037//0021-9010.61.3.374 · Cited by 68
Talking the talk on walking the walk
Holland, A., O’Connor, R. J., Thompson, A. J., Playford, E. D., & Hobart, J. C. (2006) · Journal of Neurology · doi:10.1007/s00415-006-0272-2 · Cited by 63
Self-citationRating scales for neurologists
Hobart, J. (2003) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.74.suppl_4.iv22 · Cited by 63
Self-citationA Study of the Likert Technique of Attitude Scale Construction
Ferguson, L. W. (1941) · The Journal of Social Psychology · doi:10.1080/00224545.1941.9714060 · Cited by 61
A framework relating outcomes based education and the taxonomy of educational objectives
Andrich, D. (2002) · Studies In Educational Evaluation · doi:10.1016/s0191-491x(02)00011-1 · Cited by 59
Item Selection by the Constant Process
Ferguson, G. A. (1942) · Psychometrika · doi:10.1007/bf02288601 · Cited by 57
MISUNDERSTANDING THE RASCH MODEL
Wright, B. D. (1977) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1977.tb00039.x · Cited by 57
X.—The Factorial Analysis of Multiple Item Tests
Lawley, D. N. (1944) · Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Sciences · doi:10.1017/s0080454100006440 · Cited by 46
Brogden, H. E. (1946) · Psychometrika · doi:10.1007/bf02290130 · Cited by 45
Clinimetric properties of a walking scale in peripheral neuropathy
Graham, R. C. (2006) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp.2005.081497 · Cited by 45
Skeletal muscle fatigue and physical endurance of young and old mice
Pagala, M. K. D., Ravindran, K., Namba, T., & Grob, D. (1998) · Muscle & Nerve · doi:10.1002/(sici)1097-4598(199812)21:12<1814::aid-mus40>3.0.co;2-g · Cited by 42
Exploring rating scale responsiveness: Does the total score reflect the sum of its parts?
O’Connor, R. J., Cano, S. J., Thompson, A. J., & Hobart, J. C. (2004) · Neurology · doi:10.1212/01.wnl.0000116136.22922.d6 · Cited by 39
Self-citationMeasuring quality of life in multiple sclerosis: not as simple as it sounds
Nicholl, L., Hobart, J., Cramp, F. L., & Lowe-Strong, A. S. (2005) · Multiple Sclerosis Journal · doi:10.1191/1352458505ms1235oa · Cited by 24
Self-citationESTIMATING PARAMETERS IN THE RASCH MODEL: REMOVING THE EFFECTS OF RANDOM GUESSING
Waller, M. I. (1976) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1976.tb01094.x · Cited by 21
Probability, disposition, and the inconsistency of attitudes and behavior
Duncan, O. D. (1986) · Synthese · doi:10.1007/bf00413967 · Cited by 14
Psychiatric, business, and psychological applications of fundamental measurement models
Allerup, P., Bech, P., Loldrup, D., Álvarez, P., Palacios, T. M. B., Styles, I. M., & Tenenbaum, G. (1994) · International Journal of Educational Research · doi:10.1016/0883-0355(94)90014-0 · Cited by 1
Cited by
Development of a New Patient-Reported Outcome Measure for Breast Surgery: The BREAST-Q
Pusic, A. L., Klassen, A. F., Scott, A. M., Klok, J. A., Cordeiro, P. G., & Cano, S. J. (2009) · Plastic and Reconstructive Surgery · doi:10.1097/prs.0b013e3181aee807 · Cited by 1,873
Self-citationRasch-built Overall Disability Scale (R-ODS) for immune-mediated peripheral neuropathies
van Nes, S. I., Vanhoutte, E. K., van Doorn, P. A., Hermans, M., Bakkers, M., Kuitwaard, K., Faber, C. G., & Merkies, I. S. J. (2011) · Neurology · doi:10.1212/wnl.0b013e318208824b · Cited by 381
Petrillo, J., Cano, S. J., McLeod, L. D., & Coon, C. D. (2015) · Value in Health · doi:10.1016/j.jval.2014.10.005 · Cited by 355
Self-citationRevised upper limb module for spinal muscular atrophy: Development of a new module
Mazzone, E. S., Mayhew, A., Montes, J., Ramsey, D., Fanelli, L., Young, S. D., Salazar, R., De Sanctis, R., Pasternak, A., Glanzman, A., et al. (2017) · Muscle & Nerve · doi:10.1002/mus.25430 · Cited by 347
Klassen, A. F., Cano, S. J., Schwitzer, J. A., Scott, A. M., & Pusic, A. L. (2015) · Plastic and Reconstructive Surgery · doi:10.1097/prs.0000000000000895 · Cited by 280
Self-citation
Metadata
- Authors
- Hobart, J., & Cano, S.
- Affiliations
- Peninsula College of Medicine and Dentistry (GB)
- UCL Queen Square Institute of Neurology (GB)
- Corresponding author
- Hobart, J
- Venue
- Health Technology Assessment
- Published
- 2009-02
- Volume
- 13
- Issue
- 12
- Pages
- iii, ix-x, 1
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2063923378
- Semantic Scholar
- 2f17618556f8b0f11d05f13faf9e03eb40d8876a
- Cited by
- 621
- References
- 240
- Citations by year
- 2026: 34 · 2025: 46 · 2024: 59 · 2023: 48 · 2022: 38 · 2021: 58 · 2020: 41 · 2019: 36 · 2018: 29 · 2017: 41 · 2016: 27 · 2015: 35 · 2014: 29 · 2013: 35 · 2012: 22
- FWCI
- 9.38
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Health Technology Assessment (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Health Technology Assessment programme (95/01/05)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Health Sciences › Medicine › Clinical Neurology › Multiple Sclerosis Research Studies
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Keywords
- Rasch measurement, item response theory, multiple sclerosis, rating scales, psychometric assessment, Rivermead Mobility Index, MSIS-29, test-retest reliability, Barthel Index, Functional Independence Measure, neurorehabilitation, health measurement
- MeSH terms
- Activities of Daily Living, Evaluation Studies as Topic, Humans, Models, Psychological, Multiple Sclerosis, Psychometrics, Reproducibility of Results, Logistic Models, Sickness Impact Profile, Self Efficacy, Mobility Limitation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.