The importance of patient-reported outcomes in clinical trials and strategies for future optimization
Open accessThe DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Abstract
Patient-reported outcomes (PROs) can be included in clinical trials as primary or secondary endpoints and are increasingly recognized by regulators, clinicians, and patients as valuable tools to collect patient-centered data. PROs provide unique information on the impact of a medical condition and its treatment from the patient's perspective; therefore, PROs can be included in clinical trials to ensure the impact of a trial intervention is comprehensively assessed. This review first discusses examples of how PRO endpoints have added value to clinical trial interpretation. Second, it describes the problems with current practices in designing, implementing, and reporting PRO studies, and how these problems may be addressed by complying with guidance for protocol development, selecting appropriate PRO measures to match clinically motivated PRO hypotheses, minimizing the rates of avoidable missing PRO data, analyzing and interpreting PRO data, and transparently reporting PRO findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The weaRAble-PRO study administered a selection of validated PRO measures for RA in complement to bespoke digital PRO assessments—that are validated in clinical trials”
From Digital health technologies and machine learning augment patient reported outcomes to remotely characterise rheumatoid arthritis (2024) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
SPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., Hróbjartsson, A., Mann, H., Dickersin, K., Berlin, J. A., et al. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-3-201302050-00583 · Cited by 7,363 · cited 2×
“It should be used in conjunction with the SPIRIT 2013 statement 32,33 when developing a research protocol for a trial with a primary or important secondary PRO endpoint.”
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251 · cited 2×
“It should be used in conjunction with the SPIRIT 2013 statement 32,33 when developing a research protocol for a trial with a primary or important secondary PRO endpoint.”
CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000251 · Cited by 2,296
Tannock, I. F., Osoba, D., Stockler, M. R., Ernst, D. S., Neville, A. J., Moore, M. J., Armitage, G. R., Wilson, J. J., Venner, P. M., Coppin, C. M., et al. (1996) · Journal of Clinical Oncology · doi:10.1200/jco.1996.14.6.1756 · Cited by 1,580 · cited 2×
“21,22 The PRO endpoints led to regulatory approval of mitoxantrone for this indication and wide implementation in the clinic.”
InfluentialSelf-citationReporting of Patient-Reported Outcomes in Randomized Trials: The CONSORT PRO Extension
Calvert, M., Blazeby, J., Altman, D. G., Revicki, D. A., Moher, D., Brundage, M. D., & CONSORT PRO Group, F. T. (2013) · JAMA · doi:10.1001/jama.2013.879 · Cited by 1,386 · cited 2×
“This work informed the development of a CONSORT-PRO Extension (Consolidated Standards of Reporting Trials), which can be considered the minimum reporting items for PRO endpoints (primary or key secondary) in RCTs. 73 CONSORT-PRO consists of 14 items, five of which were newly developed PRO-specific reporting recommenda…”
Self-citationReducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319 · cited 1×
“However, the issue of non-reporting of PROs is emerging and requires attention, as it is unethical to waste the funding, resources, and patients’ efforts expended in PRO data collection.(71)”
Cherny, N. I., Sullivan, R., Dafni, U., Kerst, J. M., Sobrero, A., Zielinski, C., de Vries, E. G. E., & Piccart, M. J. (2015) · Annals of Oncology · doi:10.1093/annonc/mdv249 · Cited by 818 · cited 2×
“In oncology, for example, the American Society of Clinical Oncology (ASCO)(10) and European Society for Medical Oncology (ESMO)(11) have proposed standardized approaches to evaluate clinical trial results by using scores to evaluate the Magnitude of Clinical Benefit (ESMO MCB) or the Net Health Benefit (ASCO), which i…”
Calvert, M., Kyte, D., Mercieca-Bebber, R., Slade, A., Chan, A.-W., King, M. T., and the SPIRIT-PRO Group, Hunn, A., Bottomley, A., Regnault, A., et al. (2018) · JAMA · doi:10.1001/jama.2017.21903 · Cited by 814 · cited 5×
“A second review examined the PRO content of 26 protocols of Phase III international, ovarian cancer trials published between 2000 and 2016 against a list of PRO-specific items that would eventually inform the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)-PRO Extension.(29,30) Of the 26 tr…”
InfluentialSelf-citationSchnipper, L. E., Davidson, N. E., Wollins, D. S., Blayney, D. W., Dicker, A. P., Ganz, P. A., Hoverman, J. R., Langdon, R., Lyman, G. H., Meropol, N. J., et al. (2016) · Journal of Clinical Oncology · doi:10.1200/jco.2016.68.2518 · Cited by 655 · cited 2×
“In oncology, for example, the American Society of Clinical Oncology (ASCO)(10) and European Society for Medical Oncology (ESMO)(11) have proposed standardized approaches to evaluate clinical trial results by using scores to evaluate the Magnitude of Clinical Benefit (ESMO MCB) or the Net Health Benefit (ASCO), which i…”
Revicki, D. A., Osoba, D., Fairclough, D., Barofsky, I., Berzon, R., Leidy, N. K., & Rothman, M. (2000) · Quality of Life Research · doi:10.1023/a:1008996223999 · Cited by 577
du Bois, A., Quinn, M., Thigpen, T., Vermorken, J., Avall-Lundqvist, E., Bookman, M., Bowtell, D., Brady, M., Casado, A., Cervantes, A., et al. (2005) · Annals of Oncology · doi:10.1093/annonc/mdi961 · Cited by 465
2010 Gynecologic Cancer InterGroup (GCIG) Consensus Statement on Clinical Trials in Ovarian Cancer
Stuart, G. C. E., Kitchener, H., Bacon, M., duBois, A., Friedlander, M., Ledermann, J., Marth, C., Thigpen, T., & Trimble, E. (2011) · International Journal of Gynecological Cancer · doi:10.1097/igc.0b013e31821b2568 · Cited by 425
Self-citationHandling missing data in RCTs; a review of the top medical journals
Bell, M. L., Fiero, M., Horton, N. J., & Hsu, C.-H. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-118 · Cited by 395 · cited 1×
“High rates of missing PRO data can reduce study power,(47) inflate risk of type 2 error,(48) bias interpretation,(47) and potentially undermine randomization.(49) Fielding et al determined that 18% of RCTs published in the four highest-impact medical journals between 2005 and 2006 had missing PRO data rates exceeding…”
Kluetz, P. G., Slagle, A., Papadopoulos, E. J., Johnson, L. L., Donoghue, M., Kwitkowski, V. E., Chen, W.-H., Sridhara, R., Farrell, A. T., Keegan, P., et al. (2016) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-15-2035 · Cited by 334 · cited 2×
“36 Snyder et al describe how to develop a measurement strategy for prospective labeling claims.”
Enhancing the quality and transparency of health research
Groves, T. (2008) · BMJ · doi:10.1136/bmj.a718 · Cited by 332 · cited 1×
“34”
Practical and statistical issues in missing data for longitudinal patient-reported outcomes
Bell, M. L., & Fairclough, D. L. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280213476378 · Cited by 285 · cited 3×
“It is impossible to determine the true missing data mechanism, but information about the clinical status of the patient or the reasons for missing data, particularly whether these reasons are related to the patient’s worsening illness or not, can assist researchers in deducing the underlying mechanism.(52) A review of…”
Osoba, D., Tannock, I. F., Ernst, D. S., & Neville, A. J. (1999) · Journal of Clinical Oncology · doi:10.1200/jco.1999.17.6.1654 · Cited by 224 · cited 1×
“01).(21,22) The PRO endpoints led to regulatory approval of mitoxantrone for this indication and wide implementation in the clinic.”
Clinical Trials in Recurrent Ovarian Cancer
Friedlander, M., Trimble, E., Tinker, A., Alberts, D., Avall-Lundqvist, E., Brady, M., Harter, P., Pignata, S., Pujade-Lauraine, E., Sehouli, J., et al. (2011) · International Journal of Gynecological Cancer · doi:10.1097/igc.0b013e31821bb8aa · Cited by 224
Self-citationMissing quality of life data in cancer clinical trials: serious problems and challenges
Bernhard, J., Cella, D. F., Coates, A. S., Fallowfield, L. J., Ganz, P. A., Moinpour, C. M., Mosconi, P., Osoba, D., Simes, R. J., & Hürny, C. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980315/15)17:5/7<679::aid-sim814>3.0.co;2-x · Cited by 216
Vodicka, E., Kim, K., Devine, E. B., Gnanasakthy, A., Scoggins, J. F., & Patrick, D. L. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.04.004 · Cited by 211 · cited 1×
“Consecutive reviews of ClinicalTrials.gov in 2004–2007 5 and 2007–2013 6 determined that use of PRO endpoints had increased from 14% to 27% of trials over that period.”
Patient-reported outcomes in randomized clinical trials: development of ISOQOL reporting standards
Brundage, M., Blazeby, J., Revicki, D., Bass, B., de Vet, H., Duffy, H., Efficace, F., King, M., Lam, C. L. K., Moher, D., et al. (2013) · Quality of Life Research · doi:10.1007/s11136-012-0252-1 · Cited by 207 · cited 2×
“The CONSORT-PRO extension, Calvert et al 73 The iSOQOL reporting standards, Brundage et al 72 The CONSORT-PRO extension, Calvert et al 73 The CONSORT-PRO extension, Calvert et al 73 The”
Self-citationMissing quality of life data in cancer clinical trials: serious problems and challenges
Bernhard, J., Cella, D. F., Coates, A. S., Fallowfield, L., Ganz, P. A., Moinpour, C. M., Mosconi, P., Osoba, D., Simes, J., & Hürny, C. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980315/15)17:5/7<517::aid-sim799>3.0.co;2-s · Cited by 206
Bottomley, A., Pe, M., Sloan, J., Basch, E., Bonnetain, F., Calvert, M., Campbell, A., Cleeland, C., Cocks, K., Collette, L., et al. (2016) · The Lancet Oncology · doi:10.1016/s1470-2045(16)30510-1 · Cited by 203 · cited 1×
“challenges in using analysis methods that are suited to the PRO research question, statistically correct, handle missing data appropriately, transparent to clinicians, and comparable with results of other trials in systematic reviews.(62)”
Self-citationIncorporating the patient experience into regulatory decision making in the USA, Europe, and Canada
Kluetz, P. G., O'Connor, D. J., & Soltys, K. (2018) · The Lancet Oncology · doi:10.1016/s1470-2045(18)30097-4 · Cited by 198 · cited 1×
“Increases in the use of PROs in clinical trials may be attributable to top-down encouragement from professional societies and regulatory bodies.(9) In oncology, for example, the American Society of Clinical Oncology (ASCO)(10) and European Society for Medical Oncology (ESMO)(11) have proposed standardized approaches t…”
Mesa, R. A., Gotlib, J., Gupta, V., Catalano, J. V., Deininger, M. W., Shields, A. L., Miller, C. B., Silver, R. T., Talpaz, M., Winton, E. F., et al. (2013) · Journal of Clinical Oncology · doi:10.1200/jco.2012.44.4489 · Cited by 190 · cited 2×
“Conversely, the placebo arm experienced a steady decline across all myelofibrosis symptoms over the 24 weeks.(23,24) Gnanasakthy et al have published a series of papers detailing the FDA labels that include PRO data that support regulatory approvals.”
Missing quality of life data in cancer clinical trials: serious problems and challenges
Bernhard, J., Cella, D. F., Coates, A. S., Fallowfield, L., Ganz, P. A., Moinpour, C. M., Mosconi, P., Osoba, D., Simes, J., & Hürny, C. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980315/15)17:5/7<517::aid-sim799>3.3.co;2-j · Cited by 164
Scoggins, J. F., & Patrick, D. L. (2009) · Contemporary Clinical Trials · doi:10.1016/j.cct.2009.02.005 · Cited by 158
Brundage, M., Bass, B., Davidson, J. R., Queenan, J. A., Bezjak, A., Ringash, J. G., Wilkinson, A. N., & Feldman‐Stewart, D. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9793-3 · Cited by 149 · cited 1×
“In recent years, numerous studies have scrutinized the reporting of PROs in clinical trials, using various checklists of recommended reporting items, 35,57,63–69 as summarized in Table 3.”
The CONSORT Patient-Reported Outcome (PRO) extension: implications for clinical trials and practice
Calvert, M., Brundage, M., Jacobsen, P. B., Schünemann, H. J., & Efficace, F. (2013) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-11-184 · Cited by 149 · cited 1×
“PRO findings should be published according to CONSORT-PRO and ISOQOL PRO reporting guidelines, 72,75 in a timely manner.”
Self-citationWhy are missing quality of life data a problem in clinical trials of cancer therapy?
Fairclough, D. L., Peterson, H. F., & Chang, V. T.-S. (1998) · Statistics in Medicine · Cited by 148
Efficace, F., Fayers, P. M., Pusic, A. L., Cemal, Y., Yanagawa, J., Jacobs, M., la Sala, A., Cafaro, V., Whale, K., Rees, J. R. E., et al. (2015) · Cancer · doi:10.1002/cncr.29489 · Cited by 138
A Review of Patient-Reported Outcome Labels in the United States: 2006 to 2010
Gnanasakthy, A., Mordin, M., Clark, M., DeMuro, C., Fehnel, S. E., & Copley‐Merriman, C. (2012) · Value in Health · doi:10.1016/j.jval.2011.11.032 · Cited by 129 · cited 1×
“Of these products, 71% included a PRO as the primary trial endpoint, and pain-related endpoints were most common (38%).(27) Between 2011 and 2015, 30 of the 182 (17%) new drugs approved by the FDA received PRO labeling.”
Joly, F., Vardy, J. L., Pintilie, M., & Tannock, I. F. (2007) · Annals of Oncology · doi:10.1093/annonc/mdm121 · Cited by 126
Patient-Reported Outcome Instrument Selection: Designing a Measurement Strategy
Snyder, C. F., Watson, M. E., Jackson, J. D., Cella, D. F., & Halyard, M. Y. (2007) · Value in Health · doi:10.1111/j.1524-4733.2007.00270.x · Cited by 122 · cited 2×
“Snyder et al describe how to develop a measurement strategy for prospective labeling claims.(37) This involves identifying relevant PRO domains, developing a conceptual framework around these domains, identifying approaches to measure these domains, and designing a measurement strategy based on this information.”
Fairclough, D. L., Peterson, H. F., Cella, D. F., & Bonomi, P. (1998) · Statistics in Medicine · Cited by 122
Mercieca-Bebber, R., Palmer, M. J., Brundage, M., Calvert, M., Stockler, M. R., & King, M. T. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010938 · Cited by 121 · cited 1×
“Often the missing data mechanism has a stronger impact on the results than does the proportion of missing data.(52,56) For example, if analysis methods assume that missing data are not related to the patients’ illness (eg, as with “complete case” analysis methods), when in fact the illness may have led to the data not…”
Self-citationMode of administration does not cause bias in patient-reported outcome results: a meta-analysis
Rutherford, C., Costa, D. S. J., Mercieca‐Bebber, R., Rice, H. E., Gabb, L., & King, M. T. (2015) · Quality of Life Research · doi:10.1007/s11136-015-1110-8 · Cited by 120
Self-citationAu, H., Ringash, J. G., Brundage, M., Palmer, M. K., Richardson, H., & Meyer, R. M. (2010) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/erp.10.15 · Cited by 111 · cited 4×
“First, PROs may assist clinicians and future patients to select the best treatment by providing a clearer picture of the costs and benefits of treatment.(3) Second, PRO data can enrich our understanding of the patients’ experience with unique information that could not be gained from biomedical outcomes alone, as cert…”
InfluentialVodicka, E. L., Kim, K., Devine, B., Gnanasakthy, A., Scoggins, J. F., & Patrick, D. L. (2015) · Value in Health · doi:10.1016/j.jval.2015.03.1214 · Cited by 102
Fielding, S., MacLennan, G., Cook, J. A., & Ramsay, C. R. (2008) · Trials · doi:10.1186/1745-6215-9-51 · Cited by 94 · cited 2×
“In many cases, the methods used to address missing PRO data in these trials were unclear or unsatisfactory.(50) In an update of this review (2013–2014), ~8% of RCTs published in high-impact journals used complete case analysis for PROs(51) despite the inappropriateness of this approach in many contexts.”
Systematic Evaluation of the Patient-Reported Outcome (PRO) Content of Clinical Trial Protocols
Kyte, D., Duffy, H., Fletcher, B., Gheorghe, A., Mercieca-Bebber, R., King, M., Draper, H., Ives, J., Brundage, M., Blazeby, J., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110229 · Cited by 82 · cited 2×
“28 A second review examined the PRO content of 26 protocols of Phase III international, ovarian cancer trials published between 2000 and 2016 against a list of PRO-specific items that would eventually inform the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)-PRO…”
Self-citationKyte, D. G., Ives, J., Draper, H., Keeley, T., & Calvert, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076625 · Cited by 75
Self-citationBylicki, O., Gan, H. K., Joly, F., Maillet, D., You, B., & Péron, J. (2014) · Annals of Oncology · doi:10.1093/annonc/mdu489 · Cited by 75
Guidelines for assessing quality of life in EORTC clinical trials
Young, T., de Haes, J. C. J. M., Curran, D., Fayers, P. M., & Brandberg, Y. (1999) · Cited by 73
Identifying the types of missingness in quality of life data from clinical trials
Curran, D., Bacchi, M., Schmitz, S. ‐. H., Molenberghs, G., & Sylvester, R. J. (1998) · Statistics in Medicine · Cited by 73
Planning and reporting of quality-of-life outcomes in cancer trials
Schandelmaier, S., Conen, K. L., von Elm, E. B., You, J. J., Blümle, A., Tomonaga, Y., Amstutz, A., Briel, M., Kasenda, B., Schandelmaier, S., et al. (2015) · Annals of Oncology · doi:10.1093/annonc/mdv283 · Cited by 68 · cited 3×
“Schandelmaier et al’s study was based on RCTs approved by six ethics committees from three countries, and results may therefore not represent the rate of PRO reporting internationally.(70) However, the issue of non-reporting of PROs is emerging and requires attention, as it is unethical to waste the funding, resources…”
InfluentialLuckett, T., & King, M. T. (2010) · European Journal of Cancer · doi:10.1016/j.ejca.2010.08.002 · Cited by 61 · cited 1×
“take a minimalist approach to ad hoc items, that is, avoid adding to or modifying existing PRO measures.(38)”
Self-citationA Review of Patient-Reported Outcome Labeling in the United States (2011–2015)
Gnanasakthy, A., Mordin, M., Evans, E., Doward, L. C., & DeMuro, C. (2016) · Value in Health · doi:10.1016/j.jval.2016.10.006 · Cited by 61
Gnanasakthy, A., DeMuro, C., Clark, M., Haydysch, E. E., Ma, E., & Bonthapally, V. (2016) · Journal of Clinical Oncology · doi:10.1200/jco.2015.63.6480 · Cited by 58 · cited 7×
“The remaining nine CONSORT-PRO items are elaborated from the CONSORT-2010 74 statement to specifically address PROs.”
InfluentialReporting of Patient-Reported Outcomes in Randomized Trials: The CONSORT PRO Extension
Patrick, D. L. (2013) · Value in Health · doi:10.1016/j.jval.2013.04.001 · Cited by 56
Preliminary evidence on the uptake, use and benefits of the CONSORT-PRO extension
Mercieca‐Bebber, R., Rouette, J., Calvert, M., King, M. T., McLeod, L., Holch, P., Palmer, M., & Brundage, M. (2017) · Quality of Life Research · doi:10.1007/s11136-017-1508-6 · Cited by 56 · cited 3×
“Table 3 shows that RCTs published after the release of CONSORT-PRO 73 generally had higher rates of reporting on most criteria, particularly those RCTs which cited CONSORT-PRO.”
InfluentialSelf-citationFielding, S., Ogbuagu, A., Sivasubramaniam, S., MacLennan, G., & Ramsay, C. R. (2016) · Quality of Life Research · doi:10.1007/s11136-016-1411-6 · Cited by 54
Efficace, F., Jacobs, M., Pusic, A. L., Greimel, E., Piciocchi, A., Kieffer, J. M., Gilbert, A., Fayers, P. M., & Blazeby, J. M. (2014) · European Journal of Cancer · doi:10.1016/j.ejca.2014.04.005 · Cited by 53
Mercieca‐Bebber, R., Perreca, A., King, M. T., Macann, A., Whale, K., Soldati, S., Jacobs, M., & Efficace, F. (2016) · European Journal of Cancer · doi:10.1016/j.ejca.2015.12.025 · Cited by 49
Self-citationAnalysing randomised controlled trials with missing data: Choice of approach affects conclusions
Fielding, S., Fayers, P. M., & Ramsay, C. R. (2012) · Contemporary Clinical Trials · doi:10.1016/j.cct.2011.12.002 · Cited by 47 · cited 1×
“Although methods exist to address problems with missing data statistically, 60,61 it is better practice to prevent avoidable types of missing PRO data, such as staff failing to provide the questionnaires to patients, whenever possible, and to also clearly report the rates, reasons, and handling of…”
Patient led PROMs must take centre stage in cancer research
Wilson, R. (2018) · Research Involvement and Engagement · doi:10.1186/s40900-018-0092-4 · Cited by 45 · cited 5×
“The remaining nine CONSORT-PRO items are elaborated from the CONSORT-2010 74 statement to specifically address PROs.”
InfluentialTaking patient reported outcomes centre stage in cancer research – why has it taken so long?
Selby, P. J., & Velikova, G. (2018) · Research Involvement and Engagement · doi:10.1186/s40900-018-0109-z · Cited by 43
Mercieca‐Bebber, R., Williams, D. O., Tait, M.-A., Roydhouse, J. K., Busija, L., Shunmugasundaram, C., Wilson, M. K., Langford, A., Rutherford, C., Roberts, N. A., et al. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1921-5 · Cited by 42
Self-citationMercieca‐Bebber, R., Friedlander, M. L., Kok, P., Calvert, M., Kyte, D. G., Stockler, M. R., & King, M. T. (2016) · Quality of Life Research · doi:10.1007/s11136-016-1339-x · Cited by 39 · cited 4×
“A second review examined the PRO content of 26 protocols of Phase III international, ovarian cancer trials published between 2000 and 2016 against a list of PRO-specific items that would eventually inform the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)-PRO Extension.(29,30) Of the 26 tr…”
InfluentialSelf-citationMercieca‐Bebber, R., Calvert, M., Kyte, D. G., Stockler, M. R., & King, M. T. (2017) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2017.11.009 · Cited by 35 · cited 4×
“Only two of the 20 coordinators interviewed had received PROspecific training, and two others received no PRO training whatsoever.(40) PRO training was more frequently provided by colleagues or briefly addressed at trial start-up meetings.”
InfluentialSelf-citationPost, W. J., Buijs, C., Stolk, R. P., de Vries, E. G. E., & le Cessie, S. (2009) · Quality of Life Research · doi:10.1007/s11136-009-9564-1 · Cited by 34 · cited 1×
“Although methods exist to address problems with missing data statistically, 60,61 it is better practice to prevent avoidable types of missing PRO data, such as staff failing to provide the questionnaires to patients, whenever possible, and to also clearly report the rates, reasons, and handling of…”
Mercieca‐Bebber, R., Friedlander, M. L., Calvert, M., Stockler, M. R., Kyte, D. G., Kok, P., & King, M. T. (2017) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-017-0008-3 · Cited by 34 · cited 2×
“This suggests a plausible link between the quality of PRO content of the protocol and the quality of PRO data collected, highlighting the importance of comprehensively addressing PROs in the trial protocol.(57) Palmer et al developed a classification framework for factors associated with missing PRO data.”
Self-citationStandards of reporting: the use of CONSORT PRO and CERT in individuals living with osteoporosis
Mack, D. E., Wilson, P. M., Santos, E., & Brooks, K. (2017) · Osteoporosis International · doi:10.1007/s00198-017-4249-z · Cited by 29
Palmer, M., Mercieca‐Bebber, R., King, M. T., Calvert, M., Richardson, H., & Brundage, M. (2017) · Clinical Trials · doi:10.1177/1740774517741113 · Cited by 27 · cited 1×
“).(58) Due to a majority of studies failing to report the reasons for missing PRO data, it is impossible to reliably estimate the rates of missing data attributed to these different categories.”
Self-citationAhmed, K., Kyte, D. G., Keeley, T., Efficace, F., Armes, J., Brown, J. M., Calman, L. A., Copland, C., Gavin, A., Glaser, A., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012863 · Cited by 26 · cited 1×
“A third study is in progress which will review the PRO content of cancer trials (all cancers) submitted to the UK NIHR Portfolio; 31 however, at the time of this review, the results were yet to be published.”
Self-citationScientific rigour in psycho‐oncology trials: why and how to avoid common statistical errors
Bell, M. L., Olivier, J., & King, M. T. (2012) · Psycho-Oncology · doi:10.1002/pon.3046 · Cited by 19 · cited 1×
“In an update of this review (2013–2014), ~8% of RCTs published in high-impact journals used complete case analysis for PROs(51) despite the inappropriateness of this approach in many contexts.(52,53) Furthermore, the rate of missing PRO data was unclear in 35% of studies, 10% of trials had >20% missing PRO data, and o…”
Self-citationLarkin, M. E., Lorenzi, G. M., Bayless, M., Cleary, P. A., Barnie, A., Golden, E., HITT, S. M., & GENUTH, S. M. (2012) · Clinical Trials · doi:10.1177/1740774512449532 · Cited by 15 · cited 1×
“…to feelings of burden Experienced clinical trial groups such as SWOG 45 and the Epidemiology of Diabetes Interventions and Complications group 46 have seen benefits including increased PRO compliance, improved work satisfaction, and participant retention, as a result of involving trial…”
Identifying the types of missingness in quality of life data from clinical trials
Curran, D., Bacchi, M., Schmitz, S. ‐. H., Molenberghs, G., & Sylvester, R. J. (1998) · Statistics in Medicine · Cited by 8
Why are missing quality of life data a problem in clinical trials of cancer therapy?
Fairclough, D. L., Peterson, H. F., & Chang, V. I. C. (1998) · Statistics in Medicine · Cited by 7
Kyte, D. G., Ives, J., Draper, H., Keeley, T., & Calvert, M. (2013) · Trials · doi:10.1186/1745-6215-14-s1-o56 · Cited by 3 · cited 2×
“39 An Australian study of 20 cancer clinical trial coordinators revealed that PRO administration procedures were often unclear regarding participants who were unable to complete questionnaires in English, handling participants’ family members who attempted to complete PRO measures, whether to…”
Self-citationEnhancing Nurse Contributions to SWOG Clinical Trials
Hansen, L. K., Moinpour, C. M., & Ermete, R. B. (2013) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2013.12.005 · Cited by 2 · cited 2×
“…for PRO assessment if this is not explained to them, which may contribute to feelings of burden Experienced clinical trial groups such as SWOG 45 and the Epidemiology of Diabetes Interventions and Complications group 46 have seen benefits including increased PRO compliance, improved work…”
Research works printed but not linked: 2
- [15]Food and Drug Administration Ovarian Cancer Endpoints Workshop April 26, 2006, Meeting Summary 2006 Available from: http://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/Developmen-tResources/CancerDrugs/ucm094598.pdf accessed 09 August 2016 https://www.fda.gov/downloads/AboutFDA/CentersOffices/CDER/ucm120657.pdf Accessed 14 August 2017.v
- [56]Mercieca-Bebber R Palmer MJ Brundage M Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review BMJ Open 2016 6 6 e010938 PMC4916640 27311907
Cited by
Aday, J. S., Heifets, B. D., Pratscher, S. D., Bradley, E., Rosen, R., & Woolley, J. D. (2022) · Psychopharmacology · doi:10.1007/s00213-022-06123-7 · Cited by 237
Campbell, R., Ju, A., King, M. T., & Rutherford, C. (2022) · Quality of Life Research · doi:10.1007/s11136-021-03003-z · Cited by 188
Self-citationRivera, S. C., Kyte, D. G., Aiyegbusi, O. L., Slade, A. L., McMullan, C., & Calvert, M. J. (2019) · Health and Quality of Life Outcomes · doi:10.1186/s12955-019-1220-z · Cited by 192
“metrics were drew upon the ‘pathways to research impact’ framework [6] Rivera et al.”
BackgroundMethodInfluentialSelf-citationInterventions for the management of long covid (post-covid condition): living systematic review
Zeraatkar, D., Ling, M., Kirsh, S., Jassal, T., Shahab, M., Movahed, H., Talukdar, J. R., Walch, A., Chakraborty, S. P., Turner, T., et al. (2024) · BMJ · doi:10.1136/bmj-2024-081318 · Cited by 185
“135-137 Implications Our findings suggest that offering patients with long covid a programme of CBT or a programme of physical and mental health rehabilitation will probably improve symptoms.”
Calvert, M., King, M., Mercieca-Bebber, R., Aiyegbusi, O., Kyte, D., Slade, A., Chan, A.-W., Basch, E., Bell, J., Bennett, A., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045105 · Cited by 144
Self-citation
Metadata
- Authors
- Mercieca-Bebber, R., King, M. T., Calvert, M. J., Stockler, M. R., & Friedlander, M.
- Affiliations
- The University of Sydney (AU)
- National Institute for Health and Care Research (GB)
- Corresponding author
- Friedlander, Michael
- Venue
- Patient Related Outcome Measures
- Publisher
- Informa UK Limited
- Published
- 2018-11
- Volume
- Volume 9
- Pages
- 353-367
- ISSN
- 1179-271X (ISSN-L)
- PMC
- PMC6219423
- OpenAlex
- W2899320385
- Cited by
- 659
- References
- 73
- Citations by year
- 2026: 77 · 2025: 129 · 2024: 138 · 2023: 88 · 2022: 85 · 2021: 75 · 2020: 54 · 2019: 12
- FWCI
- 4.87
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Patient Related Outcome Measures (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Dove Medical Press (Taylor and Francis Group) (submittedVersion, cc-by-nc)
- Funders
- University Hospitals Birmingham NHS Foundation Trust
- Australian Government
- National Institute for Health and Care Research
- University of Birmingham
- Cancer Australia
- Surgical Reconstruction and Microbiology Research Centre
- National Health and Medical Research Council
- Birmingham Biomedical Research Centre
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Medicine › Oncology › Cancer survivorship and care
- Keywords
- patient-reported outcomes, clinical trial endpoints, missing data, protocol design, patient-centered outcomes, regulatory guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.