Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index; 44 of the 78 sources its claims cite were checked for support.

Is science really facing a reproducibility crisis, and do we need it to?

Open access
Fanelli, Daniele · 2018 · Proceedings of the National Academy of Sciences

Abstract

Efforts to improve the reproducibility and integrity of science are typically justified by a narrative of crisis, according to which most published results are unreliable due to growing problems with research and publication practices. This article provides an overview of recent evidence suggesting that this narrative is mistaken, and argues that a narrative of epochal changes and empowerment of scientists would be more accurate, inspiring, and compelling.

Overview

CoChat Summary

TL;DR

Fanelli argues that the “science in crisis” reproducibility narrative is at least partially misguided: integrity and reproducibility problems are real but do not distort most of the literature, are heterogeneously distributed, and are not growing. He proposes a narrative of epochal change and empowerment of scientists instead.

Key findings

  • Anonymous surveys suggest on average 1–2% of scientists admit to having fabricated or falsified data at least once; the affected literature is likely smaller, at least five times smaller according to a survey among psychologists. Random audits found “scientific improprieties” in 0.28% of cancer clinical trials, problems sufficient for “for cause” investigations in 4% of FDA clinical trials from 1977–1988, and visual inspections suggested 1–2% of microbiology papers were manipulated in ways suggesting intentional fabrication.
  • P-hacking and false positives appear contained: analyses of P values in the medical literature suggested a false-discovery rate of only 14%; a broader analysis found P-hacking was common but had minor effects on meta-analytic conclusions, with stronger “evidential value” suggesting most published studies measure true effects. Low statistical power is below the recommended 80% in several disciplines but varies widely by subfield; for most findings to be false in psychology and neuroscience, one must assume tested hypotheses are correct much less than 50% of the time, which Fanelli calls unrealistic.
  • Large-scale reproducibility assessments are mixed: a “Many labs” project reported 10 of 13 psychological studies replicated consistently; an experimental economics analysis suggested at least 11 of 18 studies replicated; the largest psychological reproducibility initiative suggested reproducibility below 50%, but Fanelli argues this is likely too pessimistic because reproducibility varied by subfield, methodology, and replicator expertise, and reanalyses suggest it may be higher.
  • There is no evidence that misconduct and questionable research practices have increased: yearly ORI findings and the proportion of ORI investigations resulting in misconduct findings have not increased; retractions per retracting journal have not increased; errata and corrections have not increased; publication rate has not increased since the 1950s once coauthorship is adjusted for; and pressures to publish have not been convincingly linked to bias or misconduct.

Methods

  • This is an opinion/overview article, not a new empirical study; it synthesizes recent metaresearch evidence on misconduct, questionable research practices, P-hacking, statistical power, bias, and reproducibility.
  • It includes an illustrative Web of Science analysis (Fig. 1) counting records whose title, abstract, or keywords contain phrases such as “reproducibility crisis,” “replication crisis,” or “science in crisis,” classified by the author as endorsing, questioning, or otherwise discussing crisis narratives; the complete dataset dates back to 1933 and is in Dataset S1.
  • The main analysis is narrative argument and synthesis rather than a systematic review or formal meta-analysis.

Limitations

  • The paper states that the Nature survey on the reproducibility crisis was nonrandomized and may not accurately represent practicing scientists; direct data estimating misconduct prevalence are scarce; assumptions about true effect sizes, research bias, and prior probabilities are difficult to measure; and how reproducibility ought to be measured is subject to ongoing methodological and philosophical debate.
  • The text makes evident that this is an opinion piece with illustrative, non-systematic evidence selection; Fig. 1 is described as “merely illustrative” and excludes many articles that discuss the crisis narrative without using the searched phrases.
  • The paper does not report a formal quantitative synthesis, so the strength of its conclusions depends on the cited metaresearch literature rather than new primary analysis.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.1708272114

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

27 Valid · 2 Not valid · 15 Not checked · 27 Own results and interpretations

44 more claims · 24 with their cited sources checked

Create a free account to see all 71 claims and to verify the 34 references still open.

See all claims — free account
  • Probably not a substantial claim. 13
  • It may state two separate claims as one. 11

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 48 references

Cited by

See all 416

Metadata

Authors
Fanelli, D.
Affiliations
Indiana University Bloomington (US)
Corresponding author
Fanelli, Daniele
Venue
Proceedings of the National Academy of Sciences
Published
2018-03-12
Volume
115
Issue
11
Pages
2628-2631
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W2790858109
Cited by
416
References
48
Citations by year
2026: 26 · 2025: 41 · 2024: 39 · 2023: 56 · 2022: 51 · 2021: 71 · 2020: 57 · 2019: 54 · 2018: 20
FWCI
71.56
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • London School of Economics and Political Science Research Online (London School of Economics and Political Science) (submittedVersion)
  • London School of Economics and Political Science Theses Online (London School of Economics and Political Science) (submittedVersion)
Topics
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
replication crisis, reproducibility, research integrity, publication practices, research reliability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Is science really facing a reproducibility crisis, and do we need it to? https://app.cochat.ai/papers/10.1073/pnas.1708272114
The paper
Fanelli, D. (2018). Is science really facing a reproducibility crisis, and do we need it to? Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1708272114

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.