Measuring patient-reported outcomes: moving beyond misplaced common sense to hard science
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Interest in the patient's views of his or her illness and treatment has increased dramatically. However, our ability to appropriately measure such issues lags far behind the level of interest and need. Too often such measurement is considered to be a simple and trivial activity that merely requires the application of common sense. However, good quality measurement of patient-reported outcomes is a complex activity requiring considerable expertise and experience. This review considers the most important issues related to such measurement in the context of chronic disease and details how instruments should be developed, validated and adapted for use in additional languages. While there is often consensus on how best to undertake these activities, there is generally little evidence to support such accord. The present article questions these orthodox views and suggests alternative approaches that have been shown to be effective.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is especially surprising because patient involvement is not only recommended by many researchers,5, 6, 7, 8, 9 a few years ago it has also become a requirement if the PROM is used in medical product development.26”
From Patient involvement in the development of patient‐reported outcome measures: a scoping review (2017) · cites this paper as [5]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To capture the patient's perspective, it is essential that patients are involved in PROM development”
From Patient involvement in the development of patient‐reported outcome measures: a scoping review (2017) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1992) · PubMed · Cited by 33,716
The MOS 36-ltem Short-Form Health Survey (SF-36): I. Conceptual Framework and Item Selection
Ware, J. E., & Sherbourne, C. D. (1992) · Medical Care · doi:10.1097/00005650-199206000-00002 · Cited by 30,033
EQ-SD: a measure of health status from the EuroQol Group
Rabin, R., & Charro, F. D. (2001) · Annals of Medicine · doi:10.3109/07853890109002087 · Cited by 5,613 · cited 5×
“For too long, outdated generic HRQL measures such as the SF36, NHP and EQ-5D have been relied on in clinical studies.”
MethodInfluentialWild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S., Verjee-Lorenz, A., & Erikson, P. (2005) · Value in Health · doi:10.1111/j.1524-4733.2005.04054.x · Cited by 5,521 · cited 1×
“backward translation is the gold standard in translation methodology [47].”
MethodInternational Classification of Functioning, Disability and Health
World Health Organization (2021) · PsycTESTS Dataset · doi:10.1037/t76403-000 · Cited by 3,751
International Classification of Functioning, Disability and Health
Chard, G. (2004) · British Journal of Occupational Therapy · doi:10.1177/030802260406700101 · Cited by 3,688
Tennant, A., & Conaghan, P. G. (2007) · Arthritis Care & Research · doi:10.1002/art.23108 · Cited by 1,638
Employment and unemployment: a social-psychological analysis
Furnham, A. (1983) · Journal of Economic Psychology · doi:10.1016/0167-4870(83)90032-6 · Cited by 1,227
Employment and Unemployment: A Social-Psychological Analysis
Jahoda, M. (1982) · Cited by 1,097
Doyal, L., & Gough, I. (1991) · Bloomsbury Publishing Plc eBooks · doi:10.1007/978-1-349-21500-3 · Cited by 1,067
Hyde, M., Wiggins, R. D., Higgs, P., & Blane, D. B. (2003) · Aging & Mental Health · doi:10.1080/1360786031000101157 · Cited by 1,028 · cited 1×
“Of the measurement models described in the literature, the most widely applied QoL model is concerned with the extent to which disease and its treatment prevent an individual from meeting his or her needs [9-13].”
MethodPallant, J. F., & Tennant, A. (2007) · British Journal of Clinical Psychology · doi:10.1348/014466506x96931 · Cited by 998 · cited 3×
“Simple two-point response formats are preferable to multiple response formats [43].”
BackgroundMethodPatient-Reported Outcomes to Support Medical Product Labeling Claims: FDA Perspective
Patrick, D. L., Burke, L. B., Powers, J. H., Scott, J. A., Rock, E. P., Dawisha, S., O'Neill, R., & Kennedy, D. L. (2007) · Value in Health · doi:10.1111/j.1524-4733.2007.00275.x · Cited by 726 · cited 1×
“Indeed, the US Food and Drug Administration (FDA) prefers these types of PROs to be employed, as they appear to be uncomfortable with more subjective outcomes [34,35].”
BackgroundBergner, M., Bobbitt, R. A., Kressel, S., Pollard, W. E., Gilson, B. S., & Morris, J. R. (1976) · International Journal of Health Services · doi:10.2190/rhe0-ggh4-410w-la17 · Cited by 643
Acquadro, C., Berzon, R., Dubois, D., Leidy, N. K., Marquis, P., Revicki, D., & Rothman, M. (2003) · Value in Health · doi:10.1046/j.1524-4733.2003.65309.x · Cited by 519
The QLDS: A scale for the measurement of quality of life in depression
Hunt, S. M., & McKenna, S. P. (1992) · Health Policy · doi:10.1016/0168-8510(92)90004-u · Cited by 314 · cited 4×
“This approach argues that individuals are driven or motivated by their needs and that the fulfilment of these provides satisfaction and a good QoL [9].”
BackgroundMethodInfluentialSelf-citationTennant, A., Penta, M., Tesio, L., Grimby, G., Thonnard, J.-L., Slade, A., Lawton, G., Simone, A., Carter, J., Lundgren-Nilsson, Å., et al. (2004) · Medical Care · doi:10.1097/01.mlr.0000103529.63132.77 · Cited by 290 · cited 1×
“However, the results of differential item functioning analyses show that such comparisons are scientifically flawed, as questionnaire items work in different ways with different patient groups [24-27].”
BackgroundMcKenna, S. P., Doward, L. C., Alonso, J., Kohlmann, T., Niero, M., Prieto, L., & Wíren, L. (1999) · Quality of Life Research · doi:10.1023/a:1008987922774 · Cited by 263 · cited 1×
“If the required language versions are known from the outset, instrument development should be conducted in parallel in these countries [45,46].”
BackgroundSelf-citationThe Concept of Quality of Life
Hörnquist, J. O. (1982) · Scandinavian Journal of Social Medicine · doi:10.1177/140349488201000204 · Cited by 237
Importance of differentiating health status from quality of life
Bradley, C. (2001) · The Lancet · doi:10.1016/s0140-6736(00)03562-5 · Cited by 235 · cited 1×
“Bradley [29] argued that ‘clinicians may be misled into thinking that findings based on a [HRQL] instrument indicate that treatments do not damage QoL when all the data reveal is that treatments do not damage perceived health’ (page 7).”
Backgroundde Jong, Z., van der Heijde, D., McKenna, S. P., & Whalley, D. (1997) · Rheumatology · doi:10.1093/rheumatology/36.8.878 · Cited by 235 · cited 1×
“These stages are summarised in Table 2 [42-44].”
BackgroundSelf-citationDefining Patient-Reported Outcomes
Doward, L. C., & McKenna, S. P. (2004) · Value in Health · doi:10.1111/j.1524-4733.2004.7s102.x · Cited by 193 · cited 2×
“Measures of symptoms, activity limitations, health status, health-related quality of life (HRQL) and quality of life (QoL) completed by patients are all examples of PROMs [3,6].”
BackgroundMethodSelf-citationMeasuring Population Health: A Comparison Of Three Generic Health Status Measures
Macran, S., Weatherly, H. L. A., & Kind, P. (2003) · Medical Care · doi:10.1097/01.mlr.0000044901.57067.19 · Cited by 133 · cited 1×
“Some PROMs, such as the EQ-5D [14] and the Health Utilities Index [15], can be used to generate preference or utility assessments.”
MethodHagell, P., Hedin, P., Meads, D. M., Nyberg, L., & McKenna, S. P. (2010) · Value in Health · doi:10.1111/j.1524-4733.2009.00677.x · Cited by 125 · cited 1×
“A recent study has shown that the ‘dual panel’ methodology produces translations that are more acceptable to patients in the new country than the use of forward-backward translation [48].”
BackgroundSelf-citationEmery, M. P., Perrier, L. L., & Acquadro, C. (2005) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-3-12 · Cited by 111 · cited 2×
“Emery MP, Perrier LL, Acquadro C: Patient-Reported Outcome and Quality of Life Instruments Database (PROQOLID): frequently asked questions.”
Bjorner, J. B., & Pejtersen, J. H. (2010) · Scandinavian Journal of Public Health · doi:10.1177/1403494809352533 · Cited by 100 · cited 1×
“However, the results of differential item functioning analyses show that such comparisons are scientifically flawed, as questionnaire items work in different ways with different patient groups [24-27].”
BackgroundUnemployment: Personal and Social Consequences
Fineman, S. (1986) · Medical Entomology and Zoology · Cited by 91
Burke, L., Kennedy, D., Miskala, P., Papadopoulos, E., & Trentacosti, A. (2008) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2008.128 · Cited by 87 · cited 1×
“The common link between PROMs is that they collect information directly from the patient without interpretation by clinicians or others [3-5].”
BackgroundDevelopment of the BD-QoL: a quality of life measure specific to Behçet's disease.
Gilworth, G., Chamberlain, M. A., Bhakta, B. B., Haskard, D. O., Silman, A. J., & Tennant, A. (2004) · PubMed · Cited by 80
McTaggart-Cowan, H. M., Marra, C. A., Yang, Y., Brazier, J. E., Kopec, J. A., FitzGerald, J. M., Anis, A. H., & Lynd, L. D. (2008) · Quality of Life Research · doi:10.1007/s11136-008-9309-6 · Cited by 71
Mavranezouli, I., Brazier, J. E., Young, T. A., & Barkham, M. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9768-4 · Cited by 69
The international development of the RGHQoL: a quality of life measure for recurrent genital herpes
Doward, L. C., McKenna, S. P., Kohlmann, T., Niero, M., Patrick, D. L., Spencer, B., & Thorsen, H. (1998) · Quality of Life Research · doi:10.1023/a:1008857426633 · Cited by 64 · cited 1×
“If the required language versions are known from the outset, instrument development should be conducted in parallel in these countries [45,46].”
BackgroundSelf-citationDevelopment and validation of a needs‐based quality of life instrument for osteoarthritis
Keenan, A., McKenna, S. P., Doward, L. C., Conaghan, P. G., Emery, P., & Tennant, A. (2008) · Arthritis Care & Research · doi:10.1002/art.23714 · Cited by 61
Self-citationMcKenna, S. P., Ratcliffe, J., Meads, D. M., & Brazier, J. E. (2008) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-6-65 · Cited by 52
Self-citationThe responsiveness and validity of the CAMPHOR Utility Index
Meads, D. M., McKenna, S. P., Doughty, N., Das, C., Gin-Sing, W., Langley, J., & Pepke-Zaba, J. (2008) · European Respiratory Journal · doi:10.1183/09031936.00069708 · Cited by 50 · cited 1×
“Recently, utility valuations have been derived from responses to disease-specific QoL instruments, providing more accurate measurement of this construct [16-20].”
BackgroundSelf-citationDimensionality of the Dermatology Life Quality Index (DLQI): A Commentary
Nijsten, T. E. C., Meads, D., & McKenna, S. P. (2006) · Acta Dermato Venereologica · doi:10.2340/00015555-0075 · Cited by 43 · cited 1×
“However, the results of differential item functioning analyses show that such comparisons are scientifically flawed, as questionnaire items work in different ways with different patient groups [24-27].”
BackgroundSelf-citationDifferential Performance of SF-36 Items in Healthy Adults With and Without Functional Limitations
Horner‐Johnson, W., Krahn, G. L., Suzuki, R., Peterson, J. J., Roid, G. H., & Hall, T. A. (2010) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2009.12.015 · Cited by 34 · cited 1×
“Indeed, this measure has such poor psychometric properties that it has never proved to be a valuable instrument for showing differences between active treatments (see, for example, [37] and [38]).”
BackgroundRasch analysis of the assessment of children's hand skills in children with and without disabilities
Chien, C., Brown, T., & McDonald, R. L. (2010) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2010.09.022 · Cited by 26 · cited 1×
“However, the results of differential item functioning analyses show that such comparisons are scientifically flawed, as questionnaire items work in different ways with different patient groups [24-27].”
BackgroundGüthlin, C., & Walach, H. (2007) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.23.1.15 · Cited by 22 · cited 1×
“Indeed, this measure has such poor psychometric properties that it has never proved to be a valuable instrument for showing differences between active treatments (see, for example, [37] and [38]).”
BackgroundEffectiveness of Needs-Based Quality of Life Instruments
Doward, L. C., McKenna, S. P., & Meads, D. M. (2004) · Value in Health · doi:10.1111/j.1524-4733.2004.7s109.x · Cited by 19
Self-citationOperational Aspects of Quality-of-Life Assessment
Erickson, P., Taeuber, R. C., & Scott, J. (1995) · PharmacoEconomics · doi:10.2165/00019053-199507010-00005 · Cited by 17
Summary of Needs-Based Quality of Life Instruments
McKenna, S. P., Doward, L. C., Meads, D. M., Patrick, D. L., & Tennant, A. (2004) · Value in Health · doi:10.1111/j.1524-4733.2004.7s110.x · Cited by 13
Self-citationPMH21 IMPROVING THE MEASUREMENT PROPERTIES OF THE QUALITY OF LIFE IN DEPRESSION SCALE
McKenna, S. P., Whalley, D., & Doward, L. C. (2002) · Value in Health · doi:10.1016/s1098-3015(10)61383-6 · Cited by 6
Self-citationDoward, L. C., McKenna, S. P., Meads, D., Ratcliffe, J., Whalley, D., & Langley, P. (2002) · Value in Health · doi:10.1016/s1098-3015(10)61509-4 · Cited by 4
Self-citation
Research works printed but not linked: 9
- World Health Organization: International Classification of Impairments, Disabilities and Handicaps: A Manual of Classification Relating to Consequences of Disease. 1980, Geneva: World Health Organization
- Burke L: Acceptable evidence for pharmaceutical advertising and labeling. DIA Workshop on Pharmacoeconomics and Quality of Life Labeling and Marketing Claims, October 3, 2000, New Orleans, LA. 2000, Horsham, PA: Drug Information Association
- Doward LC, McKenna SP: Evolution of quality of life assessment. Care Management of Skin Diseases: Life Quality and Economic Impact. Edited by: Rajagopalan R, Sheretz EF, Anderson RT. 1997, New York: Marcel Dekker, 9-33.
- Hunt SM, McEwen J, McKenna SP: Measuring Health Status. 1986, London: Croom Helm
- Ware JE, Sherbourne CD: The MOS 36-Item Short-Form Health Survey (SF-36). Med Care. 1992, 30: 474-483.
- US Department of Health and Human Services, US Food and Drug Administration: Guidance for Industry: Patient-Reported Outcome Measures. Use in Medical Product Development to Support Labeling Claims. 2009, Washington, DC: US Food and Drug Administration
- European Medicines Agency, Committee for Medicinal Products for Human Use (CMHP): Reflection Paper on the Regulatory Guidance for the Use of Health-Related Quality of Life (HRQL) Measures in the Evaluation of Medicinal Products. 2005, London: European Medicines Agency
- Dixon P, Heaton J, Long A, Warburton A: Reviewing and applying the SF-36. Outcomes Briefing. 1994, 4: 3-25.
- European Medicines Agency, Committee for Medicinal Products for Human Use (CMHP): Draft Biomarkers Qualification: Guidance to Applicants. 2008, London: European Medicines Agency
Cited by
Cruchinho, P., López-Franco, M. D., Capelas, M. L., Almeida, S., Bennett, P. M., Miranda da Silva, M., Teixeira, G., Nunes, E., Lucas, P., & Gaspar, F. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s419714 · Cited by 469
“91– 120, 122– 127, 129,130 Some authors propose forward translation without back translation, 99, 114,118, 128, 131 or suggest back translation as an option.”
Patient involvement in the development of patient‐reported outcome measures: a scoping review
Wiering, B., de Boer, D., & Delnoij, D. (2017) · Health Expectations · doi:10.1111/hex.12442 · Cited by 371
Anker, S. D., Agewall, S., Borggrefe, M., Calvert, M., Jaime Caro, J., Cowie, M. R., Ford, I., Paty, J. A., Riley, J. P., Swedberg, K., et al. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu205 · Cited by 362
Tran, V.-T., Harrington, M., Montori, V. M., Barnes, C., Wicks, P., & Ravaud, P. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-109 · Cited by 259
“Patients must be involved in the process to ensure that the resulting tool is clear and relevant [11].”
BackgroundMinimal important differences for fatigue patient reported outcome measures—a systematic review
Nordin, Å., Taft, C., Lundgren-Nilsson, Å., & Dencker, A. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0167-6 · Cited by 238
“PROMs are widely used today in evaluating the effects of illness and treatment on symptoms, functioning, and other outcomes from the patient’s perspective [7].”
Metadata
- Authors
- McKenna, S. P.
- Affiliations
- Galen Research (United Kingdom) (GB)
- Corresponding author
- McKenna, Stephen P
- ORCID
- McKenna, Stephen P
- Venue
- BMC Medicine
- Published
- 2011-07-14
- Volume
- 9
- Issue
- 1
- Pages
- 86-86
- ISSN
- 1741-7015 (ISSN-L)
- PMC
- PMC3170214
- OpenAlex
- W2111629627
- Cited by
- 319
- References
- 44
- Citations by year
- 2026: 10 · 2025: 19 · 2024: 26 · 2023: 30 · 2022: 30 · 2021: 25 · 2020: 28 · 2019: 26 · 2018: 17 · 2017: 33 · 2016: 28 · 2015: 14 · 2014: 8 · 2013: 16 · 2012: 5
- FWCI
- 50.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- patient-reported outcomes, outcome measurement, chronic disease, instrument development, instrument validation, cross-cultural adaptation, patient perspectives
- MeSH terms
- Chronic Disease, Humans, Quality of Life, Surveys and Questionnaires, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.