CoChat

Feasibility study to examine discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ

Open access

The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.

Abstract

Objectives: Adding, omitting or changing prespecified outcomes can result in bias because it increases the potential for unacknowledged or post hoc revisions of the planned analyses. Journals have adopted initiatives such as requiring the prospective registration of trials and the submission of study protocols to promote the transparency of reporting in clinical trials. The main objective of this feasibility study was to document the frequency and types of outcome discrepancy between prespecified outcomes in the protocol and reported outcomes in trials submitted to The BMJ . Methods: A review of all 3156 articles submitted to The BMJ between 1 September 2013 and 30 June 2014. Trial registry entries, protocols and trial reports of randomised controlled trials published by The BMJ and a random sample of those rejected were reviewed. Editorial, peer reviewer comments and author responses were also examined to ascertain any reasons for discrepancies. Results: In the study period, The BMJ received 311 trial manuscripts, 21 of which were subsequently published by the journal. In trials published by The BMJ , 27% (89/333) of the prespecified outcomes in the protocol were not reported in the submitted paper and 11% (31/275) of reported outcomes were not prespecified. In the sample of 21 trials rejected by The BMJ , 19% (63/335) of prespecified outcomes went unreported and 14% (45/317) of reported outcomes were not prespecified. None of the reasons provided by published authors were suggestive of outcome reporting bias as the reasons were unrelated to the results. Conclusions: Mandating the prospective registration of a trial and requesting that a protocol be uploaded when submitting a trial article to a journal has the potential to promote transparency and safeguard the evidence base against outcome reporting biases as a result of outcome discrepancies. Further guidance is needed with regard to documenting reasons for outcome discrepancies.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmjopen-2015-010075

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Flow diagram of research articles submitted to The BMJ (1 September 2013 to 30 June 2014).
Figure 1 Flow diagram of research articles submitted to The BMJ (1 September 2013 to 30 June 2014).CC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 16 references

Research works printed but not linked: 1

  1. Von Elm, 2008, Selective reporting of outcomes of drug trials; comparison of study protocols and published articles, Swiss Med Wkly, 138, 197

Cited by

See all 16

Metadata

Authors
Weston, J., Dwan, K., Altman, D., Clarke, M., Gamble, C., Schroter, S., Williamson, P., & Kirkham, J.
Affiliations
  • University of Liverpool (GB)
  • University of Oxford (GB)
  • Queen's University Belfast (GB)
  • BMJ Group (United Kingdom) (GB)
Corresponding author
Kirkham, Jamie
Venue
BMJ Open
Publisher
BMJ
Published
2016-04
Volume
6
Issue
4
Pages
e010075
ISSN
2044-6055 (ISSN-L)
OpenAlex
W2338603026
Cited by
16
References
16
Citations by year
2024: 1 · 2022: 2 · 2021: 2 · 2020: 2 · 2019: 4 · 2017: 3 · 2016: 2
FWCI
2.80
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • BMJ Open (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • Research Explorer (The University of Manchester) (publishedVersion, other-oa)
  • Research Portal (Queen's University Belfast) (acceptedVersion, cc-by-nc)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • HighWire Press Open Archive (submittedVersion)
Funders
  • Health Research Board (TMRN-2014-1)
  • Cancer Research UK (16895)
  • Medical Research Council (MR/L004933/1, MR/L004933/2, MR/K025635/1)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords
selective reporting, reported outcomes, clinical trial registration, reporting transparency, randomized controlled trials, editorial policy, peer review, manuscript rejection
MeSH terms
Feasibility Studies, Periodicals as Topic, Registries, Randomized Controlled Trials as Topic, Publication Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Feasibility study to examine discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ. https://app.cochat.ai/papers/10.1136/bmjopen-2015-010075
The paper
Weston, J., Dwan, K., Altman, D., Clarke, M., Gamble, C., Schroter, S., Williamson, P., & Kirkham, J. (2016). Feasibility study to examine discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ. BMJ Open. https://doi.org/10.1136/bmjopen-2015-010075

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.