CoChat
Sign Up
Cited by 266Academic Publishing and Open Accessscientometrics and bibliometrics researchMeta-analysis and systematic reviews

Publication bias and the canonization of false facts

Open access
Nissen, Silas Boye, Magidson, Tali, Gross, Kevin, Bergstrom, Carl T·2016·eLife
DOI 10.7554/elife.21451PMC PMC5173326
Open this paper in CoChat — free account

Abstract

Science is facing a “replication crisis” in which many experimental findings cannot be replicated and are likely to be false. Does this imply that many scientific facts are false as well? To find out, we explore the process by which a claim becomes fact. We model the community’s confidence in a claim as a Markov process with successive published results shifting the degree of belief. Publication bias in favor of positive findings influences the distribution of published results. We find that unless a sufficient fraction of negative results are published, false claims frequently can become canonized as fact. Data-dredging, p-hacking, and similar behaviors exacerbate the problem. Should negative results become easier to publish as a claim approaches acceptance as a fact, however, true and false claims would be more readily distinguished. To the degree that the model reflects the real world, there may be serious concerns about the validity of purported facts in some disciplines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7554/elife.21451

Cites

66 of 66 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The self-correcting nature of science is at best torturously slow.

    “It can be shown that the”

    From Fraudulent studies are undermining the reliability of systematic reviews: on the prevalence of problematic images in preclinical depression studies (2025) · cites this paper as [57]

    The Discussion section, “What do we do about the problem of publication bias?” (page unavailable), says science can self-correct after false claims become accepted as facts. It does not establish that correction is “torturously slow”; it says inconsistencies soon begin to accumulate, without specifying how long correction takes. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • With negative results buried in file drawers, conclusions drawn from the published record can be misleading, and 'false facts' can become canonized in the literature.

    “With negative results buried in file drawers (55), conclusions drawn from the published record can be misleading, and 'false facts' can become canonized in the literature (56).”

    From Misinformation in and about science (2021) · cites this paper as [56]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Norris JR 1998 Markov Chains Cambridge University Press

Cited by

Sorted by citations

Related

Authors
Nissen, S. B., Magidson, T., Gross, K., & Bergstrom, C. T.
Affiliations
University of Copenhagen (DK)
Corresponding author
  • Gross, Kevin
  • Bergstrom, Carl T
ORCID
Venue
eLife
Publisher
eLife Sciences Publications, Ltd
Published
2016-12-20
Volume
5
ISSN
2050-084X (ISSN-L)
DOI
10.7554/elife.21451
PMC
PMC5173326
OpenAlex
W2515380863
Cited by
266
References
64
Citations by year
2026: 12 · 2025: 42 · 2024: 26 · 2023: 30 · 2022: 32 · 2021: 28 · 2020: 27 · 2019: 21 · 2018: 25 · 2017: 14 · 2016: 8
FWCI
42.86
Type
journal article
Language
en
Open access
Licence
Locations
  • eLife (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • John Templeton Foundation
  • Danish National Research Foundation
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
publication bias, replication crisis, false claims, scientific credibility, Markov model, belief updating, negative results, p-hacking, false positives
MeSH terms
Humans, Publication Bias, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review