CoChat
Sign Up
Cited by 31Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsPharmaceutical industry and healthcare

Statistical controversies in clinical research: comparison of primary outcomes in protocols, public clinical-trial registries and publications: the example of oncology trials

Open access
Perlmutter, A.S., Tran, V.-T., Dechartres, A., Ravaud, P.·2017·Annals of Oncology
DOI 10.1093/annonc/mdw682
Open this paper in CoChat — free account

Abstract

Background Protocols are often unavailable to peer-reviewers and readers. To detect outcome reporting bias (ORB), readers usually have to resort to publicly available descriptions of study design such as public clinical trial registries. We compared primary outcomes in protocols, ClinicalTrials.gov and publications of oncology trials and evaluated the use of ClinicalTrials.gov as compared with protocols in detecting discrepancies between planned and published outcomes.Method We searched for phase III oncology trials registered in ClinicalTrials.gov and published in the Journal of Clinical Oncology and New England Journal of Medicine between January 2014 and June 2015. We extracted primary outcomes reported in the protocol, ClinicalTrials.gov and the publication. First, we assessed the quality of primary outcome descriptions by using a published framework. Second, we evaluated modifications of primary outcomes between each source. Finally, we evaluated the agreement, specificity and sensitivity of detecting modifications between planned and published outcomes by using protocols or ClinicalTrials.gov.Results We included 65 trials, with 81 primary outcomes common among the 3 sources. The proportion of primary outcomes reporting all items from the framework was 73%, 22%, and 75% for protocols, ClinicalTrials.gov and publications, respectively. Eight (12%) trials presented a discrepancy between primary outcomes reported in the protocol and in the publication. Twelve (18.5%) trials presented a discrepancy between primary outcomes registered at ClinicalTrials.gov and in publications. We found a moderate agreement in detecting discrepant reporting of outcomes by using protocols or ClinicalTrials.gov [κ = 0.53, 95% confidence interval (0.25-0.81)]. Using ClinicalTrials.gov to detect discrepant reporting of outcomes showed high specificity (89.5%) but lacked sensitivity (75%) as compared with use of protocols.Conclusion In oncology trials, primary outcome descriptions in ClinicalTrials.gov are often of low quality and may not reflect what is in the protocol, thus limiting the detection of modifications between planned and published outcomes.

Integrity signals

Registry

Crossref · 10.1093/annonc/mdw682

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Perlmutter, A. S., Tran, V.-T., Dechartres, A., & Ravaud, P.
Affiliations
  • Columbia University (US)
  • Inserm (FR)
Corresponding author
Tran, V.-T.
ORCID
Venue
Annals of Oncology
Publisher
Elsevier BV
Published
2017-04
Volume
28
Issue
4
Pages
688-695
ISSN
0923-7534 (ISSN-L)
DOI
10.1093/annonc/mdw682
OpenAlex
W2564457125
Cited by
31
References
32
Citations by year
2026: 1 · 2025: 5 · 2024: 2 · 2023: 2 · 2022: 1 · 2021: 6 · 2020: 5 · 2019: 3 · 2018: 3 · 2017: 3
FWCI
2.79
Type
journal article
Language
en
Open access
Licence
Locations
  • Annals of Oncology (publishedVersion)
  • PubMed (publishedVersion)
  • HAL (submittedVersion, other-oa)
Funders
  • French National Institute for Cancer Research (SHSESP14-080)
  • ‘Equipe espoir de la Recherche’, Fondation pour la Recherche Médicale, Paris, France (DEQ20101221475)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
Keywords
selective reporting, primary outcome reporting, ClinicalTrials.gov, cancer clinical trials
MeSH terms
Humans, Medical Oncology, Registries, Research Design, Bias, Clinical Trials, Phase III as Topic, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review