Feasibility study to examine discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Objectives: Adding, omitting or changing prespecified outcomes can result in bias because it increases the potential for unacknowledged or post hoc revisions of the planned analyses. Journals have adopted initiatives such as requiring the prospective registration of trials and the submission of study protocols to promote the transparency of reporting in clinical trials. The main objective of this feasibility study was to document the frequency and types of outcome discrepancy between prespecified outcomes in the protocol and reported outcomes in trials submitted to The BMJ . Methods: A review of all 3156 articles submitted to The BMJ between 1 September 2013 and 30 June 2014. Trial registry entries, protocols and trial reports of randomised controlled trials published by The BMJ and a random sample of those rejected were reviewed. Editorial, peer reviewer comments and author responses were also examined to ascertain any reasons for discrepancies. Results: In the study period, The BMJ received 311 trial manuscripts, 21 of which were subsequently published by the journal. In trials published by The BMJ , 27% (89/333) of the prespecified outcomes in the protocol were not reported in the submitted paper and 11% (31/275) of reported outcomes were not prespecified. In the sample of 21 trials rejected by The BMJ , 19% (63/335) of prespecified outcomes went unreported and 14% (45/317) of reported outcomes were not prespecified. None of the reasons provided by published authors were suggestive of outcome reporting bias as the reasons were unrelated to the results. Conclusions: Mandating the prospective registration of a trial and requesting that a protocol be uploaded when submitting a trial article to a journal has the potential to promote transparency and safeguard the evidence base against outcome reporting biases as a result of outcome discrepancies. Further guidance is needed with regard to documenting reasons for outcome discrepancies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“two additional cohort studies published in 2008 [14] and 2016 [15] report similar prevalence for discrepancies as before.”
From COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time (2019) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Self-citationCONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
Self-citationSPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., Hróbjartsson, A., Mann, H., Dickersin, K., Berlin, J. A., et al. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-3-201302050-00583 · Cited by 7,363
Self-citationThe International Committee of Medical Journal Editors
van der Weyden, M. (2006) · Proceedings of the Workshop on Publishing for Biomedical Journal Editors and Reviewers · doi:10.2349/biij.2.4.e54-13 · Cited by 1,998
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Self-citationKirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133 · cited 1×
“This form of bias has been identified as a threat to evidence-based healthcare because clinical trial outcomes with statistically significant results are more likely to be published.(2) As a safeguard against this form of bias, current CONSORT (Consolidating Standards of Reporting Trials) guidance recommends that comp…”
BackgroundSelf-citationDwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127 · cited 2×
“When comparing trial publications with protocols, it was found that 40–62% of studies had at least one primary outcome that was changed, introduced or omitted.(4) Previous qualitative research in this field has found that the prevalence of incomplete outcome reporting is high and researchers were generally unaware of…”
BackgroundResultSelf-citationThe ClinicalTrials.gov Results Database — Update and Key Issues
Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1012065 · Cited by 838
Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444
Outcome Reporting in Industry-Sponsored Trials of Gabapentin for Off-Label Use
Vedula, S. S., Bero, L., Scherer, R. W., & Dickersin, K. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0906126 · Cited by 361
Research works printed but not linked: 1
- Von Elm, 2008, Selective reporting of outcomes of drug trials; comparison of study protocols and published articles, Swiss Med Wkly, 138, 197
Cited by
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754
Why clinical trial outcomes fail to translate into benefits for patients
Heneghan, C., Goldacre, B., & Mahtani, K. R. (2017) · Trials · doi:10.1186/s13063-017-1870-2 · Cited by 404
COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time
Goldacre, B., Drysdale, H., Dale, A., Milosevic, I., Slade, E., Hartley, P., Marston, C., Powell-Smith, A., Heneghan, C., & Mahtani, K. R. (2019) · Trials · doi:10.1186/s13063-019-3173-2 · Cited by 233
Common measures or common metrics? A plea to harmonize measurement results
de Beurs, E., Boehnke, J. R., & Fried, E. I. (2022) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2742 · Cited by 55
Selective outcome reporting in randomized clinical trials of dental implants
Sendyk, D. I., Rovai, E. S., Souza, N. V., Deboni, M. C. Z., & Pannuti, C. M. (2019) · Journal of Clinical Periodontology · doi:10.1111/jcpe.13128 · Cited by 35
Metadata
- Authors
- Weston, J., Dwan, K., Altman, D., Clarke, M., Gamble, C., Schroter, S., Williamson, P., & Kirkham, J.
- Affiliations
- University of Liverpool (GB)
- University of Oxford (GB)
- Queen's University Belfast (GB)
- BMJ Group (United Kingdom) (GB)
- Corresponding author
- Kirkham, Jamie
- ORCID
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2016-04
- Volume
- 6
- Issue
- 4
- Pages
- e010075
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC4853983
- OpenAlex
- W2338603026
- Cited by
- 16
- References
- 16
- Citations by year
- 2024: 1 · 2022: 2 · 2021: 2 · 2020: 2 · 2019: 4 · 2017: 3 · 2016: 2
- FWCI
- 2.80
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion, other-oa)
- Research Portal (Queen's University Belfast) (acceptedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Health Research Board (TMRN-2014-1)
- Cancer Research UK (16895)
- Medical Research Council (MR/L004933/1, MR/L004933/2, MR/K025635/1)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- selective reporting, reported outcomes, clinical trial registration, reporting transparency, randomized controlled trials, editorial policy, peer review, manuscript rejection
- MeSH terms
- Feasibility Studies, Periodicals as Topic, Registries, Randomized Controlled Trials as Topic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Feasibility study to examine discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ. https://app.cochat.ai/papers/10.1136/bmjopen-2015-010075
- The paper
- Weston, J., Dwan, K., Altman, D., Clarke, M., Gamble, C., Schroter, S., Williamson, P., & Kirkham, J. (2016). Feasibility study to examine discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ. BMJ Open. https://doi.org/10.1136/bmjopen-2015-010075
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
