Cited by 33Meta-analysis and systematic reviewsAcademic Publishing and Open AccessPharmaceutical industry and healthcare

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

Open access
TARG Meta-Research Group & Collaborators·2023·BMJ Open

Abstract

OBJECTIVES: Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature. DESIGN: Systematic review and meta-analyses. DATA SOURCES: Scopus and Web of Knowledge, published up to 15 December 2019. ELIGIBILITY CRITERIA: Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications. DATA EXTRACTION AND SYNTHESIS: Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results. RESULTS: We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies. CONCLUSIONS: registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable. REGISTRATION: osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmjopen-2023-076264

Cites

114 of 114 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Documentation of a trial's existence and prespecified outcomes in a publicly accessible registry helps increase transparency, decrease unnecessary duplication of research effort, facilitate identification of ongoing trials for prospective participants, and identify and deter selective reporting of study results.

    “Documentation of a trial’s existence and prespecified outcomes in a publicly accessible registry helps to increase transparency, decrease unnecessary duplication of research effort, facilitate identification of ongoing trials for prospective participants, and identify and deter selective reporting of study results.63 64”
  • Previous reviews have reported frequent discrepancies in primary outcomes defined in registries compared with those specified in protocols of randomised trials.

    “as previous reviews have reported frequent discrepancies in the primary outcomes defined in registries compared with those specified in protocols of randomised trials.64”
  • A systematic review of five studies found discrepancies in funding sources between protocols/registries and corresponding publications in a median of 45% (range 4-80%) of protocol-publication pairs.

    “A systematic review of five studies compared funding sources reported in protocols or trial registries with funding sources declared in the corresponding publications of the results. It found discrepancies in a median of 45% (range 4-80%) of protocol-publication pairs (not exclusively randomised trials).64”
  • A systematic review of articles comparing protocols and published reports found discrepancies about eligibility criteria in 25% to 57% of studies.

    “For example, a systematic review of articles comparing protocols and published reports (most being clinical trials or systematic reviews) found discrepancies about eligibility criteria in 25% to 57% of studies.64”
  • Major discrepancies in the primary outcomes designated in protocols, registries, and regulatory submissions versus final trial publications are common.

    “Furthermore, major discrepancies in the primary outcomes designated in protocols, registries, and regulatory submissions versus final trial publications are common.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
TARG Meta-Research Group & Collaborators
Corresponding author
TARG Meta-Research Group & Collaborators
Venue
BMJ Open
Publisher
BMJ
Published
2023-10
Volume
13
Issue
10
Pages
e076264
ISSN
2044-6055 (ISSN-L)
OpenAlex
W4387345483
Cited by
33
References
113
Citations by year
2026: 8 · 2025: 14 · 2024: 8 · 2023: 2 · 2022: 1
FWCI
2.26
Type
journal article
Language
en
Open access
Locations
  • BMJ Open (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (publishedVersion)
  • Explore Bristol Research (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Canadian Institutes of Health Research (Postdoctoral Fellowship)
  • Medical Research Council
  • MRC Integrative Epidemiology Unit (MC_UU_00011/7)
  • Fonds de Recherche du Québec - Santé (Postdoctoral fellowship)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
Keywords
selective reporting, research registration, prevalence, risk of bias, clinical research, protocol modification
MeSH terms
Humans, Prospective Studies, Bias, Prevalence, Publication Bias, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review