Cited by 191Health Systems, Economic Evaluations, Quality of LifeHealth Education and ValidationDelphi Technique in Research

Clinician-Reported Outcome Assessments of Treatment Benefit: Report of the ISPOR Clinical Outcome Assessment Emerging Good Practices Task Force

Open access
Powers, John H., Patrick, Donald L., Walton, Marc K., Marquis, Patrick, Cano, Stefan, Hobart, Jeremy, Isaac, Maria, Vamvakas, Spiros +3 more·2017·Value in Health

Abstract

A clinician-reported outcome (ClinRO) assessment is a type of clinical outcome assessment (COA). ClinRO assessments, like all COAs (patient-reported, observer-reported, or performance outcome assessments), are used to 1) measure patients' health status and 2) define end points that can be interpreted as treatment benefits of medical interventions on how patients feel, function, or survive in clinical trials. Like other COAs, ClinRO assessments can be influenced by human choices, judgment, or motivation. A ClinRO assessment is conducted and reported by a trained health care professional and requires specialized professional training to evaluate the patient's health status. This is the second of two reports by the ISPOR Clinical Outcomes Assessment-Emerging Good Practices for Outcomes Research Task Force. The first report provided an overview of COAs including definitions important for an understanding of COA measurement practices. This report focuses specifically on issues related to ClinRO assessments. In this report, we define three types of ClinRO assessments (readings, ratings, and clinician global assessments) and describe emerging good measurement practices in their development and evaluation. The good measurement practices include 1) defining the context of use; 2) identifying the concept of interest measured; 3) defining the intended treatment benefit on how patients feel, function, or survive reflected by the ClinRO assessment and evaluating the relationship between that intended treatment benefit and the concept of interest; 4) documenting content validity; 5) evaluating other measurement properties once content validity is established (including intra- and inter-rater reliability); 6) defining study objectives and end point(s) objectives, and defining study end points and placing study end points within the hierarchy of end points; 7) establishing interpretability in trial results; and 8) evaluating operational considerations for the implementation of ClinRO assessments used as end points in clinical trials. Applying good measurement practices to ClinRO assessment development and evaluation will lead to more efficient and accurate measurement of treatment effects. This is important beyond regulatory approval in that it provides evidence for the uptake of new interventions into clinical practice and provides justification to payers for reimbursement on the basis of the clearly demonstrated added value of the new intervention.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jval.2016.11.005

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The measurement tool for certain outcomes, particularly some PROMs, remains undefined due to the lack of agreed-upon measurement instruments in the literature.

    “Third, although outcome constructs were selected, the measurement tool for certain outcomes, particularly some PROMs, remains undefined due to the lack of agreed‐upon measurement instruments in the literature (Powers 3rd et al. 2017; Prinsen et al. 2016).”

    From 1st Global Consensus for Clinical Guidelines: Identifying a Core Outcome Set for Implant Dentistry in Edentulous Maxilla Rehabilitation (2026) · cites this paper as Powers 3rd et al. 2017

    Good Measurement Practices 1 and 4 (page unavailable) discuss context-dependent instrument choice and the need to establish content validity. They do not say that measurement tools for some PROMs remain undefined because the literature lacks agreed-upon instruments. Searches for that claim, including terms for consensus and unavailable instruments, found no supporting passage. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 44 · Cited by 191

Research works printed but not linked: 38

  1. [2]Walton MK Powers JH III Hobart J Clinical outcome assessments: conceptual foundation—Report of the ISPOR Clinical Outcomes Assessment—Emerging Good Practices for Outcomes Research Task Force Value Health 2015 18 741 52 26409600 PMC4610138
  2. [3]Aaronson N Alonso J Burnam A Assessing health status and quality-of-life instruments: attributes and review criteria Qual Life Res 2002 11 193 205 12074258
  3. [4]Sacks LV Shamsuddin HH Yasinskaya YI Scientific and regulatory reasons for delay and denial of FDA approval of initial applications for new drugs, 2000–2012 JAMA 2014 311 378 84 24449316
  4. [5]Ermisch M Bucsics A Vella Bonanno P Payers’ views of the changes arising through the possible adoption of adaptive pathways Front Pharmacol 2016 7 305 27733828 PMC5039228
  5. [9]Talbot GH Powers JH Fleming TR Progress on developing endpoints for registrational clinical trials of community-acquired bacterial pneumonia and acute bacterial skin and skin structure infections: update from the Biomarkers Consortium of the Foundation for the National Institutes of Health Clin Infect Dis 2012 55 1114 21 22744885 PMC3529608
  6. [10]Ashworth B Preliminary trial of carisoprodol in multiple sclerosis Practitioner 1964 192 540 2 14143329
  7. [11]Bohannon RW Smith MB Interrater reliability of a modified Ashworth scale of muscle spasticity Phys Ther 1987 67 206 7 3809245
  8. [12]Aronchick CA Lipshutz WH Wright SH A novel tableted purgative for colonoscopic preparation: efficacy and safety comparisons with Colyte and Fleet Phospho-Soda Gastrointest Endosc 2000 52 346 52 10968848
  9. [13]Mortimer AM Symptom rating scales and outcome in schizophrenia Br J Psychiatry Suppl 2007 50 s7 14 18019038
  10. [14]Powers JH III Das AF De Anda C Prokocimer P Clinician-reported lesion measurements in skin infection trials: definitions, reliability, and association with patient-reported pain Contemp Clin Trials 2016 50 265 72 27530088
  11. [16]Rubinstein E Cammarata S Oliphant T Wunderink R Linezolid Nosocomial Pneumonia Study Group Linezolid (PNU-100766) versus vancomycin in the treatment of hospitalized patients with nosocomial pneumonia: a randomized, double-blind, multicenter study Clin Infect Dis 2001 32 402 12 11170948
  12. [20]Kamper SJ Maher CG Mackay G Global rating of change scales: a review of strengths and weaknesses and considerations for design J Man Manip Ther 2009 17 163 70 20046623 PMC2762832
  13. [21]Moertel CG Hanley JA The effect of measuring error on the results of therapeutic trials in advanced cancer Cancer 1976 38 388 94 947531
  14. [22]Cooper CK Rochester G Komo S Powers JH Misclassification bias in clinicians’ assessments (CA) of patient outcomes and patient symptoms (PS) in antimicrobial therapy of acute exacerbations of chronic bronchitis (AECB) Presented at: 45th Interscience Conference on Antimicrobial Agents and Chemotherapy, American Society for Microbiology San Francisco, CA 2005
  15. [23]US Food and Drug Administration Issues in the design of clinical trials for systemic antibacterial drugs for the treatment of acute otitis media: a (public workshop) 2011 Available from: http://www.fda.gov/downloads/Drugs/NewsEvents/UCM273907.pdf Accessed January 3, 2017
  16. [24]Hoberman A Paradise JL Rockette HE Treatment of acute otitis media in children under 2 years of age N Engl J Med 2011 364 105 15 21226576 PMC3042231
  17. [25]Powers JH Guerrero ML Leidy NK Development of the Flu-PRO: a patient-reported outcome (PRO) instrument to evaluate symptoms of influenza BMC Infect Dis 2016 16 1 26729246 PMC4700740
  18. [27]Hadley JA Mosges R Desrosiers M Moxifloxacin five-day therapy versus placebo in acute bacterial rhinosinusitis Laryngoscope 2010 120 1057 62 20422704
  19. [28]Patrick DL Burke LB Gwaltney CJ Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2—assessing respondent understanding Value Health 2011 14 978 88 22152166
  20. [30]Felson DT Anderson JJ Boers M The American College of Rheumatology preliminary core set of disease activity measures for rheumatoid arthritis clinical trials. The Committee on Outcome Measures in Rheumatoid Arthritis Clinical Trials Arthritis Rheum 1993 36 729 40 8507213
  21. [31]Jones J Hunter D Consensus methods for medical and health services research BMJ 1995 311 376 80 7640549 PMC2550437
  22. [32]International Conference on Harmonisation of Technical Requirement for Registration of Pharmaceuticals for Human Use Statistical principles for clinical trials E9 1998 Available from: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf Accessed January 3, 2017
  23. [33]Prokocimer P De Anda C Fang E Tedizolid phosphate vs linezolid for treatment of acute bacterial skin and skin structure infections: the ESTABLISH-1 randomized trial JAMA 2013 309 559 69 23403680
  24. [34]Hobart J Cano S Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods Health Technol Assess 2009 13 iii ix x 1 177 19216837
  25. [35]Mokkink LB Terwee CB Knol DL The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content BMC Med Res Methodol 2010 10 22 20298572 PMC2848183
  26. [37]Cochrane AL Chapman PJ Oldham PD Observers’ errors in taking medical histories Lancet 1951 1 1007 9 14825885
  27. [38]Hernaez R Reliability and agreement studies: a guide for clinical investigators Gut 2015 64 1018 27 25873640
  28. [39]Cronbach LJ Gleser GC Assessing similarity between profiles Psychol Bull 1953 50 456 73 13112334
  29. [40]Beckstead JW Agreement, reliability, and bias in measurement: commentary on Bland and Altman (1986; 2010) Int J Nurs Stud 2011 48 134 5 20850746
  30. [42]Sedgwick P Limits of agreement (Bland-Altman method) BMJ 2013 346 f1630 23502707
  31. [43]Patrick DL Burke LB Powers JH Patient-reported outcomes to support medical product labeling claims: FDA perspective Value Health 2007 10 Suppl 2 S125 37 17995471
  32. [44]Pena-Casanova J Alzheimer’s Disease Assessment Scale—cognitive in clinical practice Int Psychogeriatr 1997 9 Suppl 1 105 14 9447433
  33. [45]Baldereschi M Amato MP Nencini P Cross-national interrater agreement on the clinical diagnostic criteria for dementia. WHO-PRA Age-Associated Dementia Working Group, WHO-Program for Research on Aging, Health of Elderly Program Neurology 1994 44 239 42 8309565
  34. [46]Dopfner M Steinhausen HC Coghill D Cross-cultural reliability and validity of ADHD assessed by the ADHD Rating Scale in a pan-European study Eur Child Adolesc Psychiatry 2006 15 Suppl 1 I46 55 17177016
  35. [47]Hanssen-Bauer K Gowers S Aalen OO Cross-national reliability of clinician-rated outcome measures in child and adolescent mental health services Adm Policy Ment Health 2007 34 513 8 17710527
  36. [48]Talbot GH Powers JH Hoffmann SC Developing outcomes assessments as endpoints for registrational clinical trials of antibacterial drugs: 2015 update from the Biomarkers Consortium of the Foundation for the National Institutes of Health Clin Infect Dis 2016 62 603 7 26668337 PMC4741356
  37. [50]Boucher HW Wilcox M Talbot GH Once-weekly dalbavancin versus daily conventional therapy for skin infection N Engl J Med 2014 370 2169 79 24897082
  38. [51]Corey GR Good S Jiang H Single-dose oritavancin versus 7–10 days of vancomycin in the treatment of gram-positive acute bacterial skin and skin structure infections: the SOLO II noninferiority study Clin Infect Dis 2015 60 254 62 25294250

Authors
Powers, J. H., Patrick, D. L., Walton, M. K., Marquis, P., Cano, S., Hobart, J., Isaac, M., Vamvakas, S., Slagle, A., Molsen, E., & Burke, L. B.
Affiliations
  • George Washington University (US)
  • University of Washington (US)
  • Janssen Research & Development (United States) (US)
  • Peninsula College of Medicine and Dentistry (GB)
  • European Medicines Agency (NL)
  • University of Maryland, Baltimore (US)
Corresponding author
Powers, John H.
Venue
Value in Health
Publisher
Elsevier BV
Published
2017-01
Volume
20
Issue
1
Pages
2-14
ISSN
1098-3015 (ISSN-L)
OpenAlex
W2569062514
Cited by
191
References
44
Citations by year
2026: 29 · 2025: 25 · 2024: 18 · 2023: 20 · 2022: 35 · 2021: 21 · 2020: 23 · 2019: 10 · 2018: 4 · 2017: 4 · 2015: 1
FWCI
20.18
Type
journal article
Language
en
Open access
Locations
  • Value in Health (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institutes of Health (Z99 AI999999)
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
  • Social Sciences › Education › Health Education and Validation
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
clinical outcome assessment, treatment benefit, content validity, rater reliability, measurement properties, clinical trials
MeSH terms
Documentation, Health Personnel, Health Status, Humans, Research Design, Reproducibility of Results, Outcome Assessment, Health Care, Advisory Committees

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review