Systematic Evaluation of the Patient-Reported Outcome (PRO) Content of Clinical Trial Protocols
Open accessAbstract
BACKGROUND: Qualitative evidence suggests patient-reported outcome (PRO) information is frequently absent from clinical trial protocols, potentially leading to inconsistent PRO data collection and risking bias. Direct evidence regarding PRO trial protocol content is lacking. The aim of this study was to systematically evaluate the PRO-specific content of UK National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme trial protocols. METHODS AND FINDINGS: We conducted an electronic search of the NIHR HTA programme database (inception to August 2013) for protocols describing a randomised controlled trial including a primary/secondary PRO. Two investigators independently reviewed the content of each protocol, using a specially constructed PRO-specific protocol checklist, alongside the 'Standard Protocol Items: Recommendations for Interventional Trials' (SPIRIT) checklist. Disagreements were resolved through discussion with a third investigator. 75 trial protocols were included in the analysis. Protocols included a mean of 32/51 (63%) SPIRIT recommendations (range 16-41, SD 5.62) and 11/33 (33%) PRO-specific items (range 4-18, SD 3.56). Over half (61%) of the PRO items were incomplete. Protocols containing a primary PRO included slightly more PRO checklist items (mean 14/33 (43%)). PRO protocol content was not associated with general protocol completeness; thus, protocols judged as relatively 'complete' using SPIRIT were still likely to have omitted a large proportion of PRO checklist items. CONCLUSIONS: The PRO components of HTA clinical trial protocols require improvement. Information on the PRO rationale/hypothesis, data collection methods, training and management was often absent. This low compliance is unsurprising; evidence shows existing PRO guidance for protocol developers remains difficult to access and lacks consistency. Study findings suggest there are a number of PRO protocol checklist items that are not fully addressed by the current SPIRIT statement. We therefore advocate the development of consensus-based supplementary guidelines, aimed at improving the completeness and quality of PRO content in clinical trial protocols.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A review of UK government funded trials found that among 75 protocols with patient reported outcomes, only 37% described the instrument's measurement properties.
“A review of UK government funded trials found that among 75 protocols with patient reported outcomes, only 37% described the instrument’s measurement properties.391”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [391]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
SPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., Hróbjartsson, A., Mann, H., Dickersin, K., Berlin, J. A., et al. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-3-201302050-00583 · Cited by 7,363 · see citing works
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251 · see citing works
U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research, & U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health (2006) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-4-79 · Cited by 2,856 · see citing works
Avoidable waste in the production and reporting of research evidence
Chalmers, I., & Glasziou, P. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60329-9 · Cited by 2,208 · see citing works
Avoidable Waste in the Production and Reporting of Research Evidence
Chalmers, I., & Glasziou, P. (2009) · The Lancet · doi:10.1097/aog.0b013e3181c3020d · Cited by 2,136 · see citing works
Babyak, M. A. (2004) · Psychosomatic Medicine · doi:10.1097/01.psy.0000127692.23278.a9 · Cited by 2,061 · see citing works
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · see citing works
Using quality of life measures in the clinical setting
Higginson, I. J., & Carr, A. J. (2001) · BMJ · doi:10.1136/bmj.322.7297.1297 · Cited by 982 · see citing works
Ghost Authorship in Industry-Initiated Randomised Trials
Gøtzsche, P. C., Hróbjartsson, A., Johansen, H. K., Haahr, M. T., Altman, D. G., & Chan, A.-W. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040019 · Cited by 289 · cited 1× · see citing works
“Our findings are concordant with the prevailing empirical evidence that important general methodological details are often missing from protocols [10–12,18,19].”
ResultAhmed, S., Berzon, R. A., Revicki, D. A., Lenderking, W. R., Moinpour, C. M., Basch, E., Reeve, B. B., & Wu, A. W. (2012) · Medical Care · doi:10.1097/mlr.0b013e318268aaff · Cited by 272 · see citing works
Pildal, J., Chan, A.-W., Hróbjartsson, A., Forfang, E., Altman, D. G., & Gøtzsche, P. C. (2005) · BMJ · doi:10.1136/bmj.38414.422650.8f · Cited by 209 · see citing works
Chan, A.-W., Hrobjartsson, A., Jorgensen, K. J., Gotzsche, P. C., & Altman, D. G. (2008) · BMJ · doi:10.1136/bmj.a2299 · Cited by 202 · see citing works
Calvert, M., & Freemantle, N. (2003) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.0269-4727.2003.00521.x · Cited by 121 · see citing works
Self-citationCalvert, M., Kyte, D. G., Duffy, H., Gheorghe, A., Mercieca‐Bebber, R., Ives, J., Draper, H., Brundage, M., Blazeby, J. M., & King, M. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110216 · Cited by 90 · see citing works
Self-citationDevelopment of indicators for patient-centred cancer care
Ouwens, M. M. M. T. J., Hermens, R., Hulscher, M., Vonk‐Okhuijsen, S. Y., Tjan‐Heijnen, V. C. G., Termeer, R., Marres, H. A. M., Wollersheim, H. C. H., & Grol, R. P. T. M. (2009) · Supportive Care in Cancer · doi:10.1007/s00520-009-0638-y · Cited by 85 · see citing works
Developing a guideline for clinical trial protocol content: Delphi consensus survey
Tetzlaff, J., Moher, D., & Chan, A. (2012) · Trials · doi:10.1186/1745-6215-13-176 · Cited by 70 · see citing works
Kyte, D. G., Ives, J., Draper, H., Keeley, T., & Calvert, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076625 · Cited by 59 · see citing works
Self-citationHróbjartsson, A., Pildal, J., Chan, A.-W., Haahr, M. T., Altman, D. G., & Gøtzsche, P. C. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.04.003 · Cited by 56 · see citing works
Patient-Reported Outcome Alerts
Kyte, D. G., Draper, H., & Calvert, M. (2013) · JAMA · doi:10.1001/jama.2013.277222 · Cited by 45 · see citing works
Self-citationHealth Technology Assessment in the UK
Raftery, J. P., & powell, J. A. (2013) · The Lancet · doi:10.1016/s0140-6736(13)61724-9 · Cited by 45 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Calvert, M., Kyte, D., Mercieca-Bebber, R., Slade, A., Chan, A.-W., King, M. T., and the SPIRIT-PRO Group, Hunn, A., Bottomley, A., Regnault, A., et al. (2018) · JAMA · doi:10.1001/jama.2017.21903 · Cited by 814 · see citing works
Self-citationMercieca-Bebber, R., King, M. T., Calvert, M. J., Stockler, M. R., & Friedlander, M. (2018) · Patient Related Outcome Measures · doi:10.2147/prom.s156279 · Cited by 659 · see citing works
“28 A second review examined the PRO content of 26 protocols of Phase III international, ovarian cancer trials published between 2000 and 2016 against a list of PRO-specific items that would eventually inform the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)-PRO…”
Self-citationIncreasing value and reducing waste in biomedical research: who's listening?
Moher, D., Glasziou, P., Chalmers, I., Nasser, M., Bossuyt, P. M. M., Korevaar, D. A., Graham, I. D., Ravaud, P., & Boutron, I. (2016) · The Lancet · doi:10.1016/s0140-6736(15)00307-4 · Cited by 462 · see citing works
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Kyte, D., Ives, J., Draper, H., & Calvert, M. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012281 · Cited by 46 · see citing works
Self-citationSpeich, B., Mann, E., Schönenberger, C. M., Mellor, K., Griessbach, A. N., Dhiman, P., Gandhi, P., Lohner, S., Agarwal, A., Odutayo, A., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.17651 · Cited by 41 · see citing works
Gryaznov, D., von Niederhäusern, B., Speich, B., Kasenda, B., Ojeda-Ruiz, E., Blümle, A., Schandelmaier, S., Mertz, D., Odutayo, A., Tomonaga, Y., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-053417 · Cited by 22 · see citing works
Speich, B., Odutayo, A., Peckham, N., Ooms, A., Stokes, J. R., Saccilotto, R., Gryaznov, D., von Niederhäusern, B., Copsey, B., Altman, D. G., et al. (2022) · Trials · doi:10.1186/s13063-022-06516-1 · Cited by 21 · see citing works
- Authors
- Kyte, D., Duffy, H., Fletcher, B., Gheorghe, A., Mercieca-Bebber, R., King, M., Draper, H., Ives, J., Brundage, M., Blazeby, J., & Calvert, M.
- Affiliations
- University of Birmingham (GB)
- University of Oxford (GB)
- London School of Hygiene & Tropical Medicine (GB)
- The University of Sydney (AU)
- Queen's University (CA)
- University of Bristol (GB)
- Corresponding author
- Calvert, Melanie
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-15
- Volume
- 9
- Issue
- 10
- Pages
- e110229
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4198237
- OpenAlex
- W2057467945
- Cited by
- 82
- References
- 20
- Citations by year
- 2026: 1 · 2025: 10 · 2024: 7 · 2023: 5 · 2022: 13 · 2021: 5 · 2020: 7 · 2019: 9 · 2018: 8 · 2017: 4 · 2016: 8 · 2015: 3 · 2014: 2
- FWCI
- 5.53
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Explore Bristol Research (submittedVersion)
- Funders
- National Institute for Health and Care Research (SPCR-017)
- Medical Research Council (G0800808)
- NIHR School for Primary Care Research
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- patient-reported outcomes, SPIRIT checklist, health technology assessment, randomized controlled trials, reporting guidelines
- MeSH terms
- Clinical Trials as Topic, Humans, Databases, Factual, Guideline Adherence, Checklist, Self Report, Patient Outcome Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
