CoChat
Sign Up
Cited by 1,135Health Education and ValidationHealth Systems, Economic Evaluations, Quality of LifeSurvey Methodology and Nonresponse

Content Validity—Establishing and Reporting the Evidence in Newly Developed Patient-Reported Outcomes (PRO) Instruments for Medical Product Evaluation: ISPOR PRO Good Research Practices Task Force Report: Part 2—Assessing Respondent Understanding

Open access
Patrick, Donald L., Burke, Laurie B., Gwaltney, Chad J., Leidy, Nancy Kline, Martin, Mona L., Molsen, Elizabeth, Ring, Lena·2011·Value in Health
DOI 10.1016/j.jval.2011.06.013
Open this paper in CoChat — free accountOpen PDF

Abstract

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts developers or users purport it to assess. A PRO instrument measures the concepts most relevant and important to a patient's condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Part 1 of this task force report covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Building on qualitative interviews and focus groups used to elicit concepts, cognitive interviews help developers craft items that can be understood by respondents in the target population and can ultimately confirm that the final instrument is appropriate, comprehensive, and understandable in the target population. Part 2 details: 1) the methods for conducting cognitive interviews that address patient understanding of items, instructions, and response options; and 2) the methods for tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. The task force report's two parts are meant to be read together. They are intended to offer suggestions for good practice in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jval.2011.06.013

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Only patients can determine whether the questionnaire captures health outcomes in a comprehensible and understandable manner.

    “and whether the questionnaire captures these outcomes in a comprehensible and understandable manner.12, 13, 14, 15”

    From Patient involvement in the development of patient‐reported outcome measures: a scoping review (2017) · cites this paper as [13]

    The Introduction recommends interviewing patients from the target population to determine how they understand questionnaire items, supporting the claim about comprehensibility. It also says evaluation draws on developer judgment and expert input, so it does not support the assertion that only patients can make that determination. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Patient involvement is seen as essential to ensure the content validity of the questionnaire.

    “Patient involvement is seen as essential to ensure the content validity of the questionnaire.12, 13, 14, 15”

    From Patient involvement in the development of patient‐reported outcome measures: a scoping review (2017) · cites this paper as [13]

  • Qualitative research is the necessary starting point for developing measurement tools to be used in quantitative research in order to ensure content validity.

    “several researchers stressed the importance of this approach as the necessary starting point for developing measurement tools to be used in quantitative research in order to ensure content validity [8-12].”

    From Interviewing to develop Patient-Reported Outcome (PRO) measures for clinical research: eliciting patients’ experience (2014) · cites this paper as [12]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [4]4. Tourangeau R. Cognitive science and survey methods. In: Jabine T, et al., eds. Cognitive Aspects of Survey Design: Building a Bridge Between Disciplines. Washington DC: National Academy Press, 1984.
  2. [14]14. Conway K, Patrick D. Translatability assessment. In: International Workshop on Comparative Survey Design & Implementation. Available from: http://www.csdiworkshop.org/pdf/3mc2008_ proceedings/session_01/Conway.pdf. [Accessed September 15, 2010].

Cited by

Sorted by citations

Related

Authors
Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., & Ring, L.
Affiliations
  • University of Washington (US)
  • United States Food and Drug Administration (US)
  • Brown University (US)
  • Uppsala University (SE)
Corresponding author
Patrick, Donald L.
ORCID
Venue
Value in Health
Publisher
Elsevier BV
Published
2011-12
Volume
14
Issue
8
Pages
978-988
ISSN
1098-3015 (ISSN-L)
DOI
10.1016/j.jval.2011.06.013
OpenAlex
W2158919887
Cited by
1,135
References
31
Citations by year
2026: 80 · 2025: 97 · 2024: 118 · 2023: 90 · 2022: 107 · 2021: 134 · 2020: 99 · 2019: 75 · 2018: 75 · 2017: 69 · 2016: 54 · 2015: 44 · 2014: 40 · 2013: 35 · 2012: 15
FWCI
142.48
Type
journal article
Language
en
Open access
Licence
Locations
  • Value in Health (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Education › Health Education and Validation
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
  • Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Keywords
content validity, patient-reported outcomes, cognitive interviewing, item development, item generation, focus groups, good scientific practice
MeSH terms
Documentation, European Union, Humans, Interviews as Topic, Surveys and Questionnaires, Research Design, United States, United States Food and Drug Administration, Outcome Assessment, Health Care, Focus Groups, Qualitative Research, Validation Studies as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review