The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index; 3 of the 8 sources its claims cite were checked for support.
No raw data, no science: another possible source of the reproducibility crisis
Open accessAbstract
A reproducibility crisis is a situation where many scientific studies cannot be reproduced. Inappropriate practices of science, such as HARKing, p-hacking, and selective reporting of positive results, have been suggested as causes of irreproducibility. In this editorial, I propose that a lack of raw data or data fabrication is another possible cause of irreproducibility. As an Editor-in-Chief of Molecular Brain , I have handled 180 manuscripts since early 2017 and have made 41 editorial decisions categorized as “Revise before review,” requesting that the authors provide raw data. Surprisingly, among those 41 manuscripts, 21 were withdrawn without providing raw data, indicating that requiring raw data drove away more than half of the manuscripts. I rejected 19 out of the remaining 20 manuscripts because of insufficient raw data. Thus, more than 97% of the 41 manuscripts did not present the raw data supporting their results when requested by an editor, suggesting a possibility that the raw data did not exist from the beginning, at least in some portions of these cases. Considering that any scientific study should be based on raw data, and that data storage space should no longer be a challenge, journals, in principle, should try to have their authors publicize raw data in a public database or journal site upon the publication of the paper to increase reproducibility of the published results and to increase public trust in science.
Overview
CoChat Summary
TL;DR
This editorial argues that lack of raw data—and possibly data fabrication—is another likely source of the reproducibility crisis, based on the author’s experience handling manuscripts as Editor-in-Chief of *Molecular Brain*. The author reports that when asked for raw data, more than 97% of 41 flagged manuscripts did not provide appropriate raw data, and proposes that journals require raw data deposition before publication.
Key findings
- Of 180 manuscripts handled since early 2017 to September 2019, 41 received “Revise before review” decisions requesting raw data; 21 were withdrawn without providing raw data, and of the remaining 20, 19 were rejected for insufficient raw data. Only 1 was sent for review and accepted, meaning more than 97% of the 41 did not or could not provide appropriate raw data.
- Among the 20 resubmitted with some raw data, 9 presented partial or no data, 7 had raw data that did not match the results, 11 of 18 western blotting manuscripts lacked size markers and/or entire membrane images, and 2 showed evidence of image duplications and inappropriate cuts/pastes.
- Among 40 withdrawn or rejected manuscripts, 14 were later published elsewhere; 12 of those journals require or recommend raw data upon request. The author contacted authors of the 12 papers: 10 did not respond, 1 sent raw data for only one sample per condition when each condition was supposed to have 6 samples, and 1 declined.
- Informal Twitter surveys: about 53% of 227 life-science respondents suspected more than two-thirds of withdrawn/insufficient manuscripts might have fabricated data; more than 60% of 56 researchers thought careful fabricators equal or outnumber careless ones, leading the author to roughly estimate that more than a quarter of the 180 manuscripts may include some misconduct.
Methods
- Editorial/descriptive analysis by the Editor-in-Chief of *Molecular Brain*, covering 180 manuscripts handled from early 2017 to September 2019, with 41 “Revise before review” decisions requesting raw data, absolute p-values, and multiple-test corrections.
- Counts of manuscript withdrawal, rejection, raw-data sufficiency, mismatches between raw data and results, and image problems; follow-up requests to authors of 14 later-published papers; and two informal Japanese-language Twitter surveys (227 and 56 life-science respondents).
- No formal statistical modeling or systematic sampling is reported; the analysis is descriptive and based on the editor’s caseload and online speculation.
Limitations
- The paper states that it cannot know what percentage of manuscripts had fabricated data without formal investigation, that its estimates are rough and casual speculation based on non-scientific and anecdotal episodes, and that the Twitter surveys reflect speculation or “gut feelings” rather than concrete examples.
- The paper notes alternative reasons authors may withdraw after being asked for raw data, including unwillingness to gather/upload data, data-mining concerns, or choosing journals that do not require disclosure; it also notes that rejected cases could be honest and careless mistakes, and that requests included absolute p-values and multiple-test corrections, so raw data may not be the sole reason for withdrawal.
- Evident from the text: the findings come from a single editor at one journal, use a non-random convenience sample, rely on subjective triggers such as data looking “too beautiful to be true,” and include no formal verification of fabrication or misconduct.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
3 Valid · 5 Not valid · 5 Not checked · 30 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Misconduct was found to be reported more frequently by medical/pharmacological researchers than others in this systematic review [13]”
§ No raw data, no science: another possible source of the reproducibility crisis > Absence of raw data means the absence of science
What the sources say
- Supportedcites [13] Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One. 2009;4(5):e5738. source
The Abstract states that, after controlling for survey-method differences, medical/pharmacological researchers reported misconduct more frequently than others. The Discussion likewise describes higher reporting in samples exclusively from medical, clinical and pharmacological research than in other fields or mixed samples. The claim concerns reporting, not whether those researchers actually committed more misconduct. (LLM, conf=0.94)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the authors of 34 out of the 40 manuscripts that did not provide raw data to Molecular Brain belonged to hospital or medical school in the analysis in this editorial.”
§ No raw data, no science: another possible source of the reproducibility crisis > Absence of raw data means the absence of science
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of the remaining 20 cases where the authors resubmitted the manuscripts with some raw data, 19 had insufficient data.”
- Also stated in: § Abstract · plain
“I rejected 19 out of the remaining 20 manuscripts because of insufficient raw data.”
§ No raw data, no science: another possible source of the reproducibility crisis > Raw data rarely comes out
plainn = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“have made 41 editorial decisions categorized as ‘Revise before review’, with comments asking the authors to provide raw data”
- Also stated in: § Abstract · plain
“have made 41 editorial decisions categorized as “Revise before review,” requesting that the authors provide raw data”
§ No raw data, no science: another possible source of the reproducibility crisis > Raw data rarely comes out
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Twelve journals out of those that published the 14 papers require or recommend that the authors provide raw data upon request from readers in their policies.”
§ No raw data, no science: another possible source of the reproducibility crisis > Raw data rarely comes out
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
17 more claims
Create a free account to see all 43 claims and to verify the 5 references still open.
- It may state two separate claims as one. 5
- Probably not a substantial claim. 18
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At least 11 journals across 8 publishers have now recognized the threat of paper mills through journal editorials (36,62–73).”
From Protection of the human gene research literature from contract cheating organizations known as research paper mills (2022) · cites this paper as [62]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To date, manuscripts or articles that may have been produced with assistance from paper mills has been recognised by researchers analysing unexpected literature trends (38,85,86) and/or verifiable errors (38,39,56) and by journal editors and experts describing unusual manuscript or publication features (10,36,37,62–73,82).”
From Protection of the human gene research literature from contract cheating organizations known as research paper mills (2022) · cites this paper as [62]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Unless submitted with the paper, requesting such data from the authors for confirming the findings of their study has until now usually been met with an unsatisfactory response.94”
From Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment (2023) · cites this paper as [94]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Over the past year, growing numbers of journals have begun to recognise the issue of manuscripts and publications from paper mills (73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83), including articles that analyse genes and drug treatments in human cell lines (77, 79).”
From Identification of human gene research articles with wrongly identified nucleotide sequences (2022) · cites this paper as [73]
Experimental — accuracy not yet measuredIn the citing paper
“and failure to provide convincing raw data upon request (36,62) (Table 2).”
From Protection of the human gene research literature from contract cheating organizations known as research paper mills (2022) · cites this paper as [62]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ No raw data, no science: another possible source of the reproducibility crisis > Absence of raw data means the absence of science
“According to COPE (Committee on Publication Ethics)‘s flowchart resource [10] on suspected fabricated data in a submitted manuscript”
§ No raw data, no science: another possible source of the reproducibility crisis > References
“10. Promoting research data sharing at Springer Nature: Of Schemes and Memes Blog. Available from: http://blogs.nature.com/ofschemesandmemes/2016/07/05/promoting-research-data-sharing-at-springer-nature. Cited 5 Nov 2019.”
Citations
Cites
1 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405
RetractedEstimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035
Research works printed but not linked: 1
- Cell Press STAR★Methods. Available from: https://www.cell.com/star-authors-guide. Cited 5 Nov 2019.
Cited by
RESCRIPt: Reproducible sequence taxonomy reference database management
(2021) · doi:10.1371/journal.pcbi.1009581 · Cited by 983
“On the one hand, increasing experimental and analytical 119 complexity pose mounting challenges to effective documentation and sharing of methodological 120 procedures and results [63–66].”
BackgroundGabelica, M., Bojčić, R., & Puljak, L. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.05.019 · Cited by 249
Open science saves lives: lessons from the COVID-19 pandemic
Besançon, L., Peiffer-Smadja, N., Segalas, C., Jiang, H., Masuzzo, P., Smout, C., Billy, E., Deforet, M., & Leyrat, C. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01304-y · Cited by 249
“Therefore, recent years have seen an increase in policies, from journals and institutions alike, asking researchers to share their raw data by default, except when data could not be shared for privacy reasons [15, 82].”
BackgroundInfluentialRESCRIPt: Reproducible sequence taxonomy reference database management for the massesPreprint
Robeson, M. S., O’Rourke, D. R., Kaehler, B. D., Ziemski, M., Dillon, M. R., Foster, J. T., & Bokulich, N. A. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.10.05.326504 · Cited by 162
Rosso, M., Wirz, R., Loretan, A. V., Sutter, N., da Cunha, C. T. P., Jarić, I., Würbel, H., & Voelkl, B. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104928 · Cited by 132
“…obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data. o Relevant literature: (Baker, 2015; Begley and Ioannidis, 2015; Freedman et al., 2015; Roth and Cox, 2015; Baker, 2016; Miyakawa, 2020; Smith and Lilley, 2019) 1.”
Metadata
- Authors
- Miyakawa, T.
- Affiliations
- Fujita Health University (JP)
- Corresponding author
- Miyakawa, Tsuyoshi
- ORCID
- Miyakawa, Tsuyoshi
- Venue
- Molecular Brain
- Published
- 2020-02-21
- Volume
- 13
- Issue
- 1
- Pages
- 24-24
- ISSN
- 1756-6606 (ISSN-L)
- PMC
- PMC7033918
- OpenAlex
- W3012838233
- Semantic Scholar
- 9813ab60ef54171243b592136f46458814525768
- Cited by
- 268
- References
- 14
- Citations by year
- 2026: 13 · 2025: 21 · 2024: 67 · 2023: 35 · 2022: 44 · 2021: 53 · 2020: 34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- Molecular Brain (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- replication crisis, raw data availability, data fabrication, data sharing, editorial policy, manuscript rejection, scholarly publishing, research integrity, selective reporting
- MeSH terms
- Data Analysis, Publications, Science, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: No raw data, no science: another possible source of the reproducibility crisis. https://app.cochat.ai/papers/10.1186/s13041-020-0552-2
- The paper
- Miyakawa, T. (2020). No raw data, no science: another possible source of the reproducibility crisis. Molecular Brain. https://doi.org/10.1186/s13041-020-0552-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
