Content validity–establishing and reporting the evidence in newly developed patient‐reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1 – eliciting concepts for a new PRO instrument
Open accessAbstract
The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts that developers or users purport it to assess. A PRO instrument measures the concepts most significant and relevant to a patient's condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Developing content for, and assessing respondent understanding of, newly developed PRO instruments for medical product evaluation will be discussed in this two-part ISPOR PRO Good Research Practices Task Force Report. Topics include the methods for generating items, documenting item development, coding of qualitative data from item generation, cognitive interviewing, and tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. Part 1 covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Part 2 covers the instrument development process, the assessment of patient understanding of the draft instrument using cognitive interviews and steps for instrument revision. The two parts are meant to be read together. They are intended to offer suggestions for good practices in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Patient involvement is seen as essential to ensure the content validity of the questionnaire.
“Patient involvement is seen as essential to ensure the content validity of the questionnaire.12, 13, 14, 15”
From Patient involvement in the development of patient‐reported outcome measures: a scoping review (2017) · cites this paper as [14]
Only patients can determine whether the questionnaire captures health outcomes in a comprehensible and understandable manner.
“and whether the questionnaire captures these outcomes in a comprehensible and understandable manner.12, 13, 14, 15”
From Patient involvement in the development of patient‐reported outcome measures: a scoping review (2017) · cites this paper as [14]
Qualitative research is the necessary starting point for developing measurement tools to be used in quantitative research in order to ensure content validity.
“several researchers stressed the importance of this approach as the necessary starting point for developing measurement tools to be used in quantitative research in order to ensure content validity [8-12].”
From Interviewing to develop Patient-Reported Outcome (PRO) measures for clinical research: eliciting patients’ experience (2014) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,470 · see citing works
Tong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509 · see citing works
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449 · see citing works
(1998) · Sage eBooks · Cited by 38,631 · see citing works
Handbook of Qualitative Research
Crozier, G., Denzin, N., & Lincoln, Y. (1994) · British Journal of Educational Studies · doi:10.2307/3121684 · Cited by 26,636 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808 · see citing works
Basics of Qualitative Research
Ozanne, J. L., Strauss, A., & Corbin, J. M. (1992) · Journal of Marketing Research · doi:10.2307/3172751 · Cited by 17,852 · see citing works
The Brief Psychiatric Rating Scale.
Overall, J. E., & Gorham, D. R. (1962) · Psychological Reports · doi:10.2466/pr0.1962.10.3.799 · Cited by 11,090 · see citing works
A General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · see citing works
The SAGE handbook of qualitative research
(2005) · Choice Reviews Online · doi:10.5860/choice.43-1330 · Cited by 10,173 · see citing works
Qualitative research. Introducing focus groups.
Kitzinger, J. (1995) · BMJ · doi:10.1136/bmj.311.7000.299 · Cited by 6,939 · see citing works
(1993) · Medical Entomology and Zoology · Cited by 6,226 · see citing works
Qualitative research in health care: Assessing quality in qualitative research
Mays, N. (2000) · BMJ · doi:10.1136/bmj.320.7226.50 · Cited by 3,742 · see citing works
A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method
Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004) · Science · doi:10.1126/science.1103572 · Cited by 3,422 · see citing works
The Sage handbook of qualitative research, 3rd ed.
Denzin, N. K., & Lincoln, Y. S. (2005) · Cited by 3,025 · see citing works
Ponterotto, J. G. (2005) · Journal of Counseling Psychology · doi:10.1037/0022-0167.52.2.126 · Cited by 2,725 · see citing works
Choose Your Method: A Comparison of Phenomenology, Discourse Analysis, and Grounded Theory
Starks, H., & Brown Trinidad, S. (2007) · Qualitative Health Research · doi:10.1177/1049732307307031 · Cited by 2,528 · see citing works
Dilemmas in qualitative health research
Yardley, L. (2000) · Psychology & Health · doi:10.1080/08870440008400302 · Cited by 2,310 · see citing works
Integrating response shift into health-related quality of life research: a theoretical model
Sprangers, M. A. G., & Schwartz, C. E. (1999) · Social Science & Medicine · doi:10.1016/s0277-9536(99)00045-3 · Cited by 2,228 · see citing works
Validity in qualitative research
Whittemore, R., Chase, S. K., & Mandle, C. L. (2001) · Qualitative Health Research · doi:10.1177/104973201129119299 · Cited by 2,144 · see citing works
Using narrative in social research: qualitative and quantitative approaches
Elliott, J. (2005) · Cited by 1,224 · see citing works
Using and analysing focus groups: Limitations and possibilities
Smithson, J. (2000) · International Journal of Social Research Methodology · doi:10.1080/136455700405172 · Cited by 1,160 · see citing works
Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., & Ring, L. (2011) · Value in Health · doi:10.1016/j.jval.2011.06.013 · Cited by 1,135 · see citing works
Self-citationQualitative research and content validity: developing best practices based on science and experience
Brod, M. I., Tesler, L. E., & Christensen, T. L. (2009) · Quality of Life Research · doi:10.1007/s11136-009-9540-9 · Cited by 948 · see citing works
Evaluative Criteria for Qualitative Research in Health Care: Controversies and Recommendations
Cohen, D. J., & Crabtree, B. F. (2008) · The Annals of Family Medicine · doi:10.1370/afm.818 · Cited by 836 · see citing works
Patient-Reported Outcomes to Support Medical Product Labeling Claims: FDA Perspective
Patrick, D. L., Burke, L. B., Powers, J. H., Scott, J. A., Rock, E. P., Dawisha, S., O'Neill, R., & Kennedy, D. L. (2007) · Value in Health · doi:10.1111/j.1524-4733.2007.00275.x · Cited by 726 · see citing works
Self-citationThe Social Contexts of Focus Groups
Hollander, J. A. (2004) · Journal of Contemporary Ethnography · doi:10.1177/0891241604266988 · Cited by 596 · see citing works
Kerr, C., Nixon, A., & Wild, D. J. (2010) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/erp.10.30 · Cited by 553 · see citing works
Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., Lenderking, W. R., Cella, D. F., & Basch, E. (2008) · Value in Health · doi:10.1111/j.1524-4733.2008.00470.x · Cited by 533 · see citing works
Self-citationAcquadro, C., Berzon, R. A., Dubois, D. J., Leidy, N. K., Marquis, P., Revicki, D. A., & Rothman, M. L. (2003) · Value in Health · doi:10.1046/j.1524-4733.2003.65309.x · Cited by 515 · see citing works
Self-citationPorter, R. S. (1985) · Theory and Society · doi:10.1007/bf00157532 · Cited by 491 · see citing works
PRO development: rigorous qualitative research as the crucial foundation
Lasch, K. E., Marquis, P., Vigneux, M., Abetz, L., Arnould, B., Bayliss, M., Crawford, B., & Rosa, K. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9677-6 · Cited by 461 · see citing works
Rothman, M. L., Burke, L. B., Erickson, P., Leidy, N. K., Patrick, D. L., & Petrie, C. D. (2009) · Value in Health · doi:10.1111/j.1524-4733.2009.00603.x · Cited by 433 · see citing works
Self-citationPeer review in health sciences
Taraprasad, D. (2002) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 350 · see citing works
Exploring the theoretical basis of nursing using advanced techniques of concept analysis
Morse, J. M. (1995) · Advances in Nursing Science · doi:10.1097/00012272-199503000-00005 · Cited by 303 · see citing works
Focus group research and “the patient's view”
Lehoux, P., Poland, B., & Daudelin, G. (2006) · Social Science & Medicine · doi:10.1016/j.socscimed.2006.05.016 · Cited by 278 · see citing works
Peer Review in Health Sciences
Godlee, F., & Jefferson, T. (1999) · DigitalGeorgetown (Georgetown University Library) · Cited by 264 · see citing works
Burke, L., Dl, K., Miskala, P., Papadopoulos, E., & Trentacosti, A. M. (2008) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2008.128 · Cited by 87 · see citing works
Self-citationAnger, J. T., Nissim, H. A., Le, T. X., Smith, A. L., Lee, U. J., Sarkisian, C. A., Litwin, M. S., Raz, S., Rodríguez, L. V., & Maliski, S. L. (2011) · Neurourology and Urodynamics · doi:10.1002/nau.21004 · Cited by 65 · see citing works
Ungar, W. J., Mirabelli, C., Cousins, M. A., & Boydell, K. (2006) · Social Science & Medicine · doi:10.1016/j.socscimed.2006.06.016 · Cited by 45 · see citing works
Enhancing the Usefulness of Qualitative Inquiry: Gaps, Directions, and Responsibilities
Morse, J. M. (2002) · Qualitative Health Research · doi:10.1177/1049732302238752 · Cited by 15 · see citing works
The meaning and form of occupational therapy as experienced by women with psychoses
Ivarsson, A., Söderback, I., & Ternestedt, B. (2002) · Scandinavian Journal of Caring Sciences · doi:10.1046/j.1471-6712.2002.00064.x · Cited by 14 · see citing works
Interviews with cancer fatigue patients
Lasch, K., Jernigan, K. A., Scott, J. A., & Piault‐Louis, E. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2009.27.15_suppl.e17577 · Cited by 1 · see citing works
Flagged first, then by citations
2 of the 47 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [30]30. Mays N, Pope C. Assessing quality in qualitative research. BMJ 2000; 320:50 -2.
Cited by
Sorted by citations
Terwee, C. B., Prinsen, C. A. C., Chiarotto, A., Westerman, M. J., Patrick, D. L., Alonso, J., Bouter, L. M., de Vet, H. C. W., & Mokkink, L. B. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1829-0 · Cited by 2,348 · see citing works
Self-citationPatrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., & Ring, L. (2011) · Value in Health · doi:10.1016/j.jval.2011.06.013 · Cited by 1,135 · see citing works
Self-citationde Souza, J. A., Yap, B. J., Wroblewski, K., Blinder, V. S., Araújo, F. S., Hlubocky, F. J., Nicholas, L. H., O’Connor, J. M., Brockstein, B., Ratain, M. J., et al. (2016) · Cancer · doi:10.1002/cncr.30369 · Cited by 932 · see citing works
The development of a financial toxicity patient‐reported outcome in cancer: The COST measure
de Souza, J. A., Yap, B. J., Hlubocky, F. J., Wroblewski, K., Ratain, M. J., Cella, D. F., & Daugherty, C. K. (2014) · Cancer · doi:10.1002/cncr.28814 · Cited by 695 · see citing works
Cappelleri, J. C., Lundy, J. J., & Hays, R. D. (2014) · Clinical Therapeutics · doi:10.1016/j.clinthera.2014.04.006 · Cited by 605 · see citing works
Biomarkers and surrogate endpoints in clinical trials
Fleming, T. R., & Powers, J. H. (2012) · Statistics in Medicine · doi:10.1002/sim.5403 · Cited by 546 · see citing works
Powers, J. H., Guerrero, M. L., Leidy, N. K., Fairchok, M. P., Rosenberg, A., Hernández, A., Stringer, S. M., Schofield, C. M., Rodríguez‐Zulueta, P., Kim, K. K., et al. (2015) · BMC Infectious Diseases · doi:10.1186/s12879-015-1330-0 · Cited by 504 · see citing works
Self-citationMatza, L. S., Patrick, D. L., Riley, A. W., Alexander, J. J., Rajmil, L., Pleil, A. M., & Bullinger, M. (2013) · Value in Health · doi:10.1016/j.jval.2013.04.004 · Cited by 478 · see citing works
Self-citationFaraone, S. V., Rostain, A. L., Blader, J., Busch, B., Childress, A. C., Connor, D. F., & Newcorn, J. H. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12899 · Cited by 400 · see citing works
Patient involvement in the development of patient‐reported outcome measures: a scoping review
Wiering, B., de Boer, D., & Delnoij, D. (2017) · Health Expectations · doi:10.1111/hex.12442 · Cited by 365 · see citing works
Electronic patient‐reported outcome systems in oncology clinical practice
Bennett, A. V., Jensen, R. E., & Basch, E. (2012) · CA A Cancer Journal for Clinicians · doi:10.3322/caac.21150 · Cited by 356 · see citing works
Walton, M. K., Powers, J. H., Hobart, J. C., Patrick, D. L., Marquis, P., Vamvakas, S., Isaac, M., Molsen, E., Cano, S., & Burke, L. B. (2015) · Value in Health · doi:10.1016/j.jval.2015.08.006 · Cited by 310 · see citing works
Self-citationChiarotto, A., Maxwell, L. J., Terwee, C. B., Wells, G. A., Tugwell, P., & Ostelo, R. W. (2016) · Physical Therapy · doi:10.2522/ptj.20150420 · Cited by 290 · see citing works
Klassen, A. F., Cano, S., Schwitzer, J. A., Scott, A., & Pusic, A. L. (2014) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000000895 · Cited by 287 · see citing works
Turner-Bowker, D. M., Lamoureux, R. E., Stokes, J., Litcher-Kelly, L., Galipeau, N., Yaworsky, A., Solomon, J., & Shields, A. L. (2018) · Value in Health · doi:10.1016/j.jval.2017.11.014 · Cited by 233 · see citing works
Khanna, R., Zou, G., D’Haens, G., Feagan, B. G., Sandborn, W. J., Vandervoort, M. K., Rolleri, R. L., Bortey, E. B., Paterson, C. A., Forbes, W. P., et al. (2014) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.13001 · Cited by 243 · see citing works
Greenhalgh, J., Dalkin, S. M., Gooding, K., Gibbons, E. J., Wright, J. M., Meads, D. M., Black, N. A., Valderas, J. M., & Pawson, R. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05020 · Cited by 238 · see citing works
Zannad, F., Garcia, A. A., Anker, S. D., Armstrong, P. W., Calvo, G., Cleland, J. G. F., Cohn, J. N., Dickstein, K., Domanski, M. J., Ekman, I., et al. (2013) · European Journal of Heart Failure · doi:10.1093/eurjhf/hft095 · Cited by 231 · see citing works
The BODY-Q: A Patient-Reported Outcome Instrument for Weight Loss and Body Contouring Treatments
Klassen, A. F., Cano, S., Alderman, A. K., Soldin, M. G., Thoma, A., Robson, S., Kaur, M. N., Papas, A., Laeken, N. V., Taylor, V. H., et al. (2016) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000000665 · Cited by 221 · see citing works
Taylor, A. M., Phillips, K., Patel, K. V., Turk, D. C., Dworkin, R. H., Beaton, D., Clauw, D. J., Gignac, M. A. M., Markman, J. D., Williams, D. A., et al. (2016) · Pain · doi:10.1097/j.pain.0000000000000577 · Cited by 208 · see citing works
Self-citationBrédart, A., Marrel, A., Abetz-Webb, L., Lasch, K., & Acquadro, C. (2014) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-12-15 · Cited by 195 · see citing works
Harris, K. K., Dawson, J., Gibbons, E. J., Lim, C., Beard, D. J., Fitzpatrick, R. M., & Price, A. D. F. (2016) · Patient Related Outcome Measures · doi:10.2147/prom.s97774 · Cited by 194 · see citing works
Powers, J. H., Patrick, D. L., Walton, M. K., Marquis, P., Cano, S., Hobart, J., Isaac, M., Vamvakas, S., Slagle, A., Molsen, E., et al. (2017) · Value in Health · doi:10.1016/j.jval.2016.11.005 · Cited by 191 · see citing works
Self-citationResponse scale selection in adult pain measures: results from a literature review
Safikhani, S., Gries, K. S., Trudeau, J., Reasner, D. S., Rüdell, K., Coons, S. J., Bush, E. N., Hanlon, J. T., Abraham, L., & Vernon, M. (2018) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-018-0053-6 · Cited by 168 · see citing works
Jones, E., Wainwright, T. W., Foster, J., Smith, J., Middleton, R., & Francis, N. K. (2014) · Annals of The Royal College of Surgeons of England · doi:10.1308/003588414x13824511649571 · Cited by 160 · see citing works
Development and Psychometric Evaluation of the FACE-Q Scales for Patients Undergoing Rhinoplasty
Klassen, A. F., Cano, S., East, C. A., Baker, S. B., Badia, L., Schwitzer, J. A., & Pusic, A. L. (2015) · JAMA Facial Plastic Surgery · doi:10.1001/jamafacial.2015.1445 · Cited by 162 · see citing works
Wolowacz, S. E., Briggs, A. H., Belozeroff, V., Clarke, P., Doward, L. C., Goeree, R. A., Lloyd, A. J., & Norman, R. (2016) · Value in Health · doi:10.1016/j.jval.2016.06.001 · Cited by 159 · see citing works
Schougaard, L. M. V., Larsen, L. P., Jessen, A., Sidenius, P., Dørflinger, L., de Thurah, A. L., & Hjøllund, N. H. (2016) · Quality of Life Research · doi:10.1007/s11136-015-1207-0 · Cited by 158 · see citing works
Chang, E. M., Gillespie, E. F., & Shaverdian, N. (2019) · Patient Related Outcome Measures · doi:10.2147/prom.s178344 · Cited by 155 · see citing works
Johnston, B. C., Patrick, D. L., Devji, T., Maxwell, L. J., Bingham, C. O., Beaton, D., Boers, M., Briel, M., Busse, J. W., Carrasco‐Labra, A., et al. (2019) · doi:10.1002/9781119536604.ch18 · Cited by 146 · see citing works
Self-citationKlassen, A. F., Kaur, M. N., Johnson, N., Kreukels, B. P. C., McEvenue, G., Morrison, S. D., Mullender, M. G., Poulsen, L., Özer, M., Rowe, W., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-025435 · Cited by 136 · see citing works
Chiarotto, A., Ostelo, R., Boers, M., & Terwee, C. B. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.11.005 · Cited by 136 · see citing works
Calvert, M., King, M. T., Mercieca‐Bebber, R., Aiyegbusi, O. L., Kyte, D. G., Slade, A. L., Chan, A., Basch, E., Bell, J. A., Bennett, A. V., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045105 · Cited by 140 · see citing works
Self-citationNew Trial Designs and Potential Therapies for Pulmonary Artery Hypertension
Gomberg‐Maitland, M., Bull, T. M., Saggar, R., Barst, R. J., Elgazayerly, A., Fleming, T. R., Grimminger, F., Rainisio, M., Stewart, D. J., Stockbridge, N., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.10.026 · Cited by 134 · see citing works
Leidy, N. K., Murray, L. T., Monz, B. U., Nelsen, L. M., Goldman, M. A., Jones, P., Dansie, E. J., & Sethi, S. (2014) · Respiratory Research · doi:10.1186/s12931-014-0124-z · Cited by 129 · see citing works
Self-citationMcCaffrey, N., Bradley, S. L., Ratcliffe, J., & Currow, D. C. (2016) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2016.02.012 · Cited by 131 · see citing works
Klassen, A. F., Grant, C., Barr, R. D., Brill, H., de Camargo, O. K., Ronen, G. M., Samaan, M. C., Mondal, T. K., Cano, S., Schlatman, A., et al. (2014) · Child Care Health and Development · doi:10.1111/cch.12207 · Cited by 133 · see citing works
EQ-5D-Y-5L: developing a revised EQ-5D-Y with increased response categories
Kreimeier, S., Åström, M., Burström, K., Egmar, A., Gusi, N., Herdman, M., Kind, P., Pérez-Sousa, M. Á., & Greiner, W. (2019) · Quality of Life Research · doi:10.1007/s11136-019-02115-x · Cited by 126 · see citing works
Haywood, K. L., Mars, T. S., Potter, R., Patel, S., Matharu, M., & Underwood, M. (2018) · Cephalalgia · doi:10.1177/0333102417731348 · Cited by 113 · see citing works
Riff, K. W. Y. W., Tsangaris, E., Goodacre, T., Forrest, C. R., Pusic, A. L., Cano, S., & Klassen, A. F. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015467 · Cited by 115 · see citing works
Patient-reported Symptoms of Tenosynovial Giant Cell Tumors
Gelhorn, H. L., Tong-Starksen, S., McQuarrie, K., Vernon, C., Hanlon, J. T., Maclaine, G. D. H., Lenderking, W. R., Ye, X., Speck, R. M., Lackman, R. D., et al. (2016) · Clinical Therapeutics · doi:10.1016/j.clinthera.2016.03.008 · Cited by 113 · see citing works
Qualitative Methods Used to Generate Questionnaire Items: A Systematic Review
Ricci, L., Lanfranchi, J., Lemétayer, F., Rotonda, C., Guillemin, F., Coste, J., & Spitz, É. (2018) · Qualitative Health Research · doi:10.1177/1049732318783186 · Cited by 113 · see citing works
Howells, L., Chalmers, J. R., Gran, S., Ahmed, A. I., Apfelbacher, C., Burton, T., Howie, L., Lawton, S., Ridd, M. J., Rogers, N. K., et al. (2019) · British Journal of Dermatology · doi:10.1111/bjd.18780 · Cited by 106 · see citing works
Cheng, L. J., Tan, R. L.-Y., & Luo, N. (2021) · Value in Health · doi:10.1016/j.jval.2021.02.003 · Cited by 111 · see citing works
Klassen, A. F., Cano, S., Schwitzer, J. A., Baker, S. B., CARRUTHERS, A. J., Carruthers, J., Chapas, A. M., & Pusic, A. L. (2016) · JAMA Dermatology · doi:10.1001/jamadermatol.2016.0018 · Cited by 107 · see citing works
Checklist to operationalize measurement characteristics of patient-reported outcome measures
Francis, D. O., McPheeters, M. L., Noud, M., Penson, D. F., & Feurer, I. D. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0307-4 · Cited by 107 · see citing works
Aldhouse, N. V. J., Kitchen, H., Knight, S. L., Macey, J., Nunes, F. P., Dutronc, Y., Mesinkovska, N. A., Ko, J., King, B., & Wyrwich, K. W. (2020) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-020-00240-7 · Cited by 105 · see citing works
The impact of hyperhidrosis on patients' daily life and quality of life: a qualitative investigation
Kamudoni, P., Mueller, B., Halford, J. C. G., Schouveller, A., Stacey, B., & Salek, S. (2017) · Health and Quality of Life Outcomes · doi:10.1186/s12955-017-0693-x · Cited by 103 · see citing works
What Matters to Patients With Cleft Lip and/or Palate
Riff, K. W. Y. W., Tsangaris, E., Goodacre, T., Forrest, C. R., Lawson, J. L., Pusic, A. L., & Klassen, A. F. (2017) · The Cleft Palate-Craniofacial Journal · doi:10.1177/1055665617732854 · Cited by 100 · see citing works
Gater, A., Rofail, D., Tolley, C., Marshall, C., Abetz‐Webb, L., Zarit, S. H., & Berardo, C. G. (2014) · Schizophrenia Research and Treatment · doi:10.1155/2014/368215 · Cited by 102 · see citing works
Klassen, A. F., Ziolkowski, N. I., Mundy, L. R., Miller, H. C., McIlvride, A., DiLaura, A., Fish, J. S., & Pusic, A. L. (2018) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000001672 · Cited by 96 · see citing works
Clinical outcome assessment in clinical trials of chronic pain treatments
Patel, K. V., Amtmann, D., Jensen, M. P., Smith, S. M., Veasley, C., & Turk, D. C. (2021) · PAIN Reports · doi:10.1097/pr9.0000000000000784 · Cited by 96 · see citing works
Patient Reported Outcomes in Rheumatoid Arthritis Clinical Trials
Orbai, A., & Bingham, C. O. (2015) · Current Rheumatology Reports · doi:10.1007/s11926-015-0501-8 · Cited by 95 · see citing works
Patient-reported outcomes in meta-analyses – Part 1: assessing risk of bias and combining outcomes
Johnston, B. C., Patrick, D. L., Busse, J. W., Schünemann, H. J., Agarwal, A., & Guyatt, G. H. (2013) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-11-109 · Cited by 95 · see citing works
Self-citationRaghu, G., Ghazipura, M., Fleming, T. R., Aronson, K. I., Behr, J., Brown, K. M., Flaherty, K. R., Kazerooni, E. A., Maher, T. M., Richeldi, L., et al. (2024) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202312-2213so · Cited by 98 · see citing works
Thompson, D. A., Iannaccone, A., Ali, R. R., Arshavsky, V. Y., Audo, I. S., Bainbridge, J. W., Besirli, C. G., Birch, D. G., Branham, K. E. H., Cideciyan, A. V., et al. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.7.2 · Cited by 94 · see citing works
Lee, E. H., Klassen, A. F., Cano, S., Nehal, K. S., & Pusic, A. L. (2018) · British Journal of Dermatology · doi:10.1111/bjd.16671 · Cited by 94 · see citing works
Mannix, S., Skalicky, A. M., Buse, D. C., Desai, P., Sapra, S. J., Ortmeier, B. G., Widnell, K. L., & Hareendran, A. (2016) · Health and Quality of Life Outcomes · doi:10.1186/s12955-016-0542-3 · Cited by 89 · see citing works
Wiering, B., de Boer, D., & Delnoij, D. M. J. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2582-8 · Cited by 89 · see citing works
Carrière, M. E., Mokkink, L. B., Tyack, Z. F., Westerman, M. J., Pijpe, A., Pleat, J., van de Kar, A. L., Brown, J. N., de Vet, H. C. W., & van Zuijlen, P. P. M. (2022) · Quality of Life Research · doi:10.1007/s11136-022-03244-6 · Cited by 91 · see citing works
Engström, M. S., Leksell, J. K., Johansson, U., & Guðbjörnsdóttir, S. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010249 · Cited by 87 · see citing works
Tsangaris, E., Riff, K. W. Y. W., Goodacre, T., Forrest, C. R., Dreise, M. M., Sykes, J. M., de Chalain, T., Harman, K., O'Mahony, A. M., Pusic, A. L., et al. (2017) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000001305 · Cited by 87 · see citing works
Tutorial on how to calculating content validity of scales in medical research
Madadizadeh, F., & Bahariniya, S. (2023) · Perioperative Care and Operating Room Management · doi:10.1016/j.pcorm.2023.100315 · Cited by 97 · see citing works
Value to Whom? The Patient Voice in the Value Discussion
Perfetto, E. M., Oehrlein, E. M., Boutin, M., Reid, S. M., & Gascho, E. (2017) · Value in Health · doi:10.1016/j.jval.2016.11.014 · Cited by 83 · see citing works
Staunton, H., Willgoss, T., Nelsen, L. M., Burbridge, C., Sully, K., Rofail, D., & Arbuckle, R. (2019) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-019-0100-y · Cited by 81 · see citing works
McCollister, D. H., Shaffer, S., Badesch, D. B., Filusch, A., Hunsche, E., Schüler, R., Wiklund, I. E., & Peacock, A. J. (2016) · Respiratory Research · doi:10.1186/s12931-016-0388-6 · Cited by 80 · see citing works
Fastenau, J. M., Kolotkin, R. L., Fujioka, K., Alba, M., Canovatchel, W. J., & Traina, S. (2019) · Clinical Obesity · doi:10.1111/cob.12309 · Cited by 78 · see citing works
Qualitative validation of the FACIT-Fatigue scale in systemic lupus erythematosus
Kosinski, M. R., Gajria, K., Fernandes, A., & Cella, D. F. (2013) · Lupus · doi:10.1177/0961203313476360 · Cited by 79 · see citing works
Measurement challenges in shared decision making: putting the ‘patient’ in patient‐reported measures
Barr, P. J., & Elwyn, G. J. (2015) · Health Expectations · doi:10.1111/hex.12380 · Cited by 78 · see citing works
Evidence-Based Interventions for Adult Aural Rehabilitation: That Was Then, This Is Now
Maidment, D. W., Henshaw, H., Heffernan, E., & Ferguson, M. (2019) · Seminars in Hearing · doi:10.1055/s-0038-1676784 · Cited by 76 · see citing works
Flythe, J. E., Powell, J. D., Poulton, C. J., Westreich, K. D., Handler, L., Reeve, B. B., & Carey, T. S. (2015) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2015.05.020 · Cited by 75 · see citing works
Measuring respiratory symptoms in clinical trials of COPD: reliability and validity of a daily diary
Leidy, N. K., Sexton, C., Jones, P., Notte, S. M., Monz, B. U., Nelsen, L. M., Goldman, M. H., Murray, L. T., & Sethi, S. (2014) · Thorax · doi:10.1136/thoraxjnl-2013-204428 · Cited by 73 · see citing works
Self-citationCrossnohere, N. L., Brundage, M., Calvert, M., King, M. T., Reeve, B. B., Thorner, E. D., Wu, A. W., & Snyder, C. F. (2020) · Quality of Life Research · doi:10.1007/s11136-020-02625-z · Cited by 77 · see citing works
FACE-Q Eye Module for Measuring Patient-Reported Outcomes Following Cosmetic Eye Treatments
Klassen, A. F., Cano, S., Grotting, J. C., Baker, S. B., Carruthers, J., CARRUTHERS, A. J., Laeken, N. V., Sykes, J. M., Schwitzer, J. A., & Pusic, A. L. (2016) · JAMA Facial Plastic Surgery · doi:10.1001/jamafacial.2016.1018 · Cited by 70 · see citing works
The PROactive innovative conceptual framework on physical activity
Dobbels, F., de Jong, C., Drost, E., Elberse, J., Feridou, C., Jacobs, L., Rabinovich, R., Frei, A., Puhan, M. A., de Boer, W. I., et al. (2014) · European Respiratory Journal · doi:10.1183/09031936.00004814 · Cited by 67 · see citing works
Sibéoni, J., Verneuil, L., Manolios, É., & Révah‐Levy, A. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01099-4 · Cited by 70 · see citing works
Patient-Reported Outcome Measures in Lymphedema: A Systematic Review and COSMIN Analysis
Beelen, L. M., van Dishoeck, A., Tsangaris, E., Coriddi, M. R., Dayan, J. H., Pusic, A. L., Klassen, A. F., & Vasilic, D. (2020) · Annals of Surgical Oncology · doi:10.1245/s10434-020-09346-0 · Cited by 72 · see citing works
Tools for assessing the scalability of innovations in health: a systematic review
Charif, A. B., Zomahoun, H. T. V., Gogovor, A., Samri, M. A., Massougbodji, J., Wolfenden, L., Ploeg, J., Zwarenstein, M., Milat, A., Rhéault, N., et al. (2022) · Health Research Policy and Systems · doi:10.1186/s12961-022-00830-5 · Cited by 71 · see citing works
A randomized crossover trial of elamipretide in adults with primary mitochondrial myopathy
Karaa, A., Haas, R., Goldstein, A. C., Vockley, J. E., & Cohen, B. H. (2020) · Journal of Cachexia Sarcopenia and Muscle · doi:10.1002/jcsm.12559 · Cited by 68 · see citing works
Mokkink, L. B., Herbelet, S., Tuinman, P. R., & Terwee, C. B. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111879 · Cited by 96 · see citing works
Characterizing mobility from the prosthetic limb user’s perspective
Hafner, B. J., Morgan, S. J., Abrahamson, D. C., & Amtmann, D. (2015) · Prosthetics and Orthotics International · doi:10.1177/0309364615579315 · Cited by 66 · see citing works
Drumond, N., van Riet‐Nales, D. A., Karapinar‐Çarkit, F., & Stegemann, S. (2017) · International Journal of Pharmaceutics · doi:10.1016/j.ijpharm.2017.02.029 · Cited by 66 · see citing works
Vaillancourt, S., Seaton, M. B., Schull, M. J., Cheng, A. H. Y., Beaton, D., Laupacis, A., & Dainty, K. N. (2017) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.05.034 · Cited by 65 · see citing works
Development of the BODY-Q Chest Module Evaluating Outcomes following Chest Contouring Surgery
Klassen, A. F., Kaur, M. N., Poulsen, L., Fielding, C., Geerards, D., van de Grift, T. C., Hoogbergen, M. M., Juhl, C. B., Lorenzen, M. M., McEvenue, G., et al. (2018) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000004978 · Cited by 66 · see citing works
Jackson, K., Parthan, A., Lauher-Charest, M., Broderick, L. E., Law, N., & Barnett, C. (2022) · Neurology and Therapy · doi:10.1007/s40120-022-00408-x · Cited by 64 · see citing works
Stevens, K. J., & Palfreyman, S. J. (2012) · Value in Health · doi:10.1016/j.jval.2012.08.2204 · Cited by 64 · see citing works
Jafree, S. R., Zakar, R., Zakar, M. Z., & Fischer, F. (2015) · BMC Health Services Research · doi:10.1186/s12913-015-1252-y · Cited by 62 · see citing works
Patient-Reported Outcome Measures
Poulsen, L., McEvenue, G., Klassen, A. F., Hoogbergen, M. M., Sørensen, J. A., & Pusic, A. L. (2018) · Clinics in Plastic Surgery · doi:10.1016/j.cps.2018.08.003 · Cited by 61 · see citing works
Patient Reported Outcome Measures (PROMs) in Inflammatory Bowel Disease: New Data
Bojic, D., Bodger, K., & Travis, S. P. L. (2016) · Journal of Crohn s and Colitis · doi:10.1093/ecco-jcc/jjw187 · Cited by 60 · see citing works
Patient-reported outcome measures for acute and chronic pain
Pogatzki-Zahn, E. M., Schnabel, K., & Kaiser, U. (2019) · Current Opinion in Anaesthesiology · doi:10.1097/aco.0000000000000780 · Cited by 59 · see citing works
A Systematic Review of Patient-Reported Outcome Measures Used in Adult Burn Research
Griffiths, C., Guest, E., White, P., Gaskin, E., Rumsey, N., Pleat, J., & Harcourt, D. M. (2016) · Journal of Burn Care & Research · doi:10.1097/bcr.0000000000000474 · Cited by 60 · see citing works
Patient-reported fatigue and its impact on patients with systemic lupus erythematosus
Sterling, K., Gallop, K., Swinburn, P., Flood, E. M., French, A., Sawah, S. A., Iikuni, N., Naegeli, A. N., & Nixon, A. (2013) · Lupus · doi:10.1177/0961203313511554 · Cited by 58 · see citing works
Louis, E. J., Ramos-Goñi, J. M., Cuervo, J., Kopylov, U., Acosta, M. B., McCartney, S. A., Rosenfeld, G. A., Bettenworth, D., Hart, A., Novak, K. L., et al. (2020) · Patient · doi:10.1007/s40271-019-00407-5 · Cited by 58 · see citing works
Qualitative Methods and Patient-Reported Outcomes
Cheng, K. K. F., & Clark, A. M. (2017) · International Journal of Qualitative Methods · doi:10.1177/1609406917702983 · Cited by 57 · see citing works
Leidy, N. K., Murray, L. T., Jones, P., & Sethi, S. (2014) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201309-305oc · Cited by 57 · see citing works
Self-citationAssessing Outcomes in Body Contouring
Klassen, A. F., Cano, S., Scott, A., Tsangaris, E., & Pusic, A. L. (2014) · Clinics in Plastic Surgery · doi:10.1016/j.cps.2014.06.004 · Cited by 57 · see citing works
Klaassen, R. J., Barrowman, N. J., Merelles‐Pulcini, M., Vichinsky, E. P., Sweeters, N., Kirby-Allen, M. A., Neufeld, E. J., Kwiatkowski, J. L., Wu, J. K., Vickars, L. M., et al. (2013) · British Journal of Haematology · doi:10.1111/bjh.12631 · Cited by 56 · see citing works
Development and validation of the Cholinergic Urticaria Quality‐of‐Life Questionnaire (CholU‐QoL)
Ruft, J., Asady, A., Staubach, P., Casale, T. B., Sussmann, G., Zuberbier, T., Maurer, M., Weller, K., & Altrichter, S. (2018) · Clinical and Experimental Allergy · doi:10.1111/cea.13102 · Cited by 45 · see citing works
Gounder, M. M., Maddux, L., Paty, J. A., & Atkinson, T. M. (2019) · Cancer · doi:10.1002/cncr.32555 · Cited by 56 · see citing works
Development of the symptoms and impacts questionnaire for Crohn's disease and ulcerative colitis
Dulai, P. S., Jairath, V., Khanna, R., Ma, C., McCarrier, K. P., Martin, M. L., Parker, C. E., Morris, J. C., Feagan, B. G., & Sandborn, W. J. (2020) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.15726 · Cited by 56 · see citing works
Self-citation
- Authors
- Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., & Ring, L.
- Affiliations
- University of Washington (US)
- United States Food and Drug Administration (US)
- Brown University (US)
- Uppsala University (SE)
- Corresponding author
- Patrick, Donald L.
- Venue
- Value in Health
- Publisher
- Elsevier BV
- Published
- 2011-12
- Volume
- 14
- Issue
- 8
- Pages
- 967-977
- ISSN
- 1098-3015 (ISSN-L)
- OpenAlex
- W2128536774
- Cited by
- 1,095
- References
- 47
- Citations by year
- 2026: 87 · 2025: 96 · 2024: 113 · 2023: 89 · 2022: 91 · 2021: 119 · 2020: 98 · 2019: 78 · 2018: 74 · 2017: 63 · 2016: 59 · 2015: 43 · 2014: 37 · 2013: 32 · 2012: 13
- FWCI
- 67.99
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Value in Health (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Social Sciences › Education › Health Education and Validation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- content validity, patient-reported outcomes, item generation, cognitive interviewing, regulatory submissions, patient understanding, good scientific practice
- MeSH terms
- Documentation, European Union, Humans, Interviews as Topic, Surveys and Questionnaires, Research Design, United States, United States Food and Drug Administration, Outcome Assessment, Health Care, Focus Groups, Qualitative Research, Validation Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review