The impact of fraudulent and irreproducible data to the translational research crisis – solutions and implementation
Open accessAbstract
One of the aims of basic neuroscience research is ultimately the development of therapeutics to cure diseases. Funders granting money to research institutions increasingly express interest into how their financial resources are used and look for successful translation in clinical practice. Disappointingly, many findings that started out promising in basic research projects and phase I trials did not live up to the promise of therapeutic efficacy in later phase II or III trials. An inordinately high amount of time and money is thus spent on research that does not always have the required human impact. Potential reasons for these problems are numerous. Although research misconduct occurs and contributes to this shortcoming, it is not the only important factor. Frequently, basic science results turn out to be irreproducible. Irreproducibility, outside of malfeasance, is multifactorial and can include poor experimental design, conduct, statistical analysis, reporting standards, and conceptual flaws. Further confounding problems include an insufficient transferability of animal to human physiology, as well as intersubject group variability, for example, sexual dimorphisms. While the causes of poor data reproducibility are therefore numerous, equally there are many groups that can contribute to improvements in how basic science is reported. Here, we will review how the Journal of Neurochemistry can contribute to increasing the value of preclinical and translational research. image Despite a vast amount of very promising basic research findings, these failed to successfully translate into the clinical practice so far. The reasons for this ‘data reproducibility crisis’ are numerous, for example, rooting in insufficient experimental design, conceptual flaws, incorrect statistical planning and evaluation, incomplete model system that do not adequately reproduce the human physiology, and further reasons discussed in this Review with the aim to present practical solutions that can be implemented by researchers, journals editors, and reviewers. We will also explain measures the Journal of Neurochemistry have implemented to overcome these issues and weaknesses in preclinical research. These includes adherence to the ARRIVE ( www.nc3rs.org) guidelines, NINDS standards (doi: 10.1038/nature11556), and The Transparency and Openness Promotion Guidelines (TOP) Committee guidelines (https://cos.io/top/#TOP). This article is part of the 60th Anniversary special issue.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The misidentification of cell lines is a stubborn problem in the biomedical sciences, contributing to errors, false conclusions and irreproducible experiments.
“The misidentification of cell lines is a stubborn problem in the biomedical sciences, contributing to the growing concerns about errors, false conclusions and irreproducible experiments [1, 2].”
From The ghosts of HeLa: How cell line misidentification contaminates the scientific literature (2017) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
Retraction of an interpretation
Murch, S. H., Anthony, A., Casson, D. H., Malik, M., Berelowitz, M., Dhillon, A. P., Thomson, M. A., Valentine, A., Davies, S. E., & Walker-Smith, J. A. (2004) · The Lancet · doi:10.1016/s0140-6736(04)15715-2 · Cited by 321 · see citing works
RetractedPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
Immunization with amyloid-β attenuates Alzheimer-disease-like pathology in the PDAPP mouse
Schenk, D., Barbour, R., Dunn, W., Gordon, G., Grajeda, H., Guido, T., Hu, K., Huang, J., Johnson-Wood, K., Khan, K., et al. (1999) · Nature · doi:10.1038/22124 · Cited by 3,160 · see citing works
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · see citing works
Misconduct accounts for the majority of retracted scientific publications
Fang, F. C., Steen, R. G., & Casadevall, A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1212247109 · Cited by 1,232 · see citing works
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196 · see citing works
Erroneous analyses of interactions in neuroscience: A problem of significance
Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2886 · Cited by 864 · see citing works
Taylor, L. E., Swerdfeger, A. L., & Eslick, G. D. (2014) · Vaccine · doi:10.1016/j.vaccine.2014.04.085 · Cited by 583 · see citing works
Design, power, and interpretation of studies in the standard murine model of ALS
Scott, S., Kranz, J. E., Cole, J., Lincecum, J. M., Thompson, K., Kelly, N., Bostrom, A., Theodoss, J., Al‐Nakhala, B. M., Vieira, F. G., et al. (2008) · Amyotrophic Lateral Sclerosis · doi:10.1080/17482960701856300 · Cited by 504 · see citing works
Reproducible Research Practices and Transparency across the Biomedical Literature
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002333 · Cited by 310 · see citing works
A resource for cell line authentication, annotation and quality control
Yu, M., Selvaraj, S. K., Liang-Chu, M. M. Y., Aghajani, S., Busse, M., Yuan, J., Lee, G., Peale, F., Klijn, C., Bourgon, R., et al. (2015) · Nature · doi:10.1038/nature14397 · Cited by 243 · cited 1× · see citing works
“The procedures suggest simple measures to detect and avoid cross-contamination, and provide a framework for the annotation of cell lines that links to short tandem repeat and single-nucleotide polymorphism profiles (Yu et al. 2015).”
MethodHolman, C., Piper, S. K., Grittner, U., Diamantaras, A. A., Kimmelman, J., Siegerink, B., & Dirnagl, U. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002331 · Cited by 114 · see citing works
The Lancet's two days to bury bad news
Deer, B. (2011) · BMJ · doi:10.1136/bmj.c7001 · Cited by 50 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Llovera G., Roth S., Plesnila N., Veltkamp R. and Liesz A. (2014) Modeling stroke in mice: permanent coagulation of the distal middle cerebral artery. J. Vis. Exp. (89): e51729. DOI: 10.3791/51729. · DOI
- Murphy S. P. and Murphy A. N. (2010) Pre-clinical systematic review. J. Neurochem. 115, 805.
- White A. T. and Murphy A. N. (2010) Administration of thiazolidinediones for neuroprotection in ischemic stroke: a pre-clinical systematic review. J. Neurochem. 115, 845-853.
Cited by
Sorted by citations
The ghosts of HeLa: How cell line misidentification contaminates the scientific literature
Horbach, S. P. J. M., & Halffman, W. (2017) · PLOS ONE · doi:10.1371/journal.pone.0186281 · Cited by 167 · see citing works
- Authors
- Schulz, J. B., Cookson, M. R., & Hausmann, L.
- Affiliations
- Forschungszentrum Jülich (DE)
- National Institutes of Health (US)
- Universitätsklinikum Aachen (DE)
- Corresponding author
- Schulz, Jörg B.
- Venue
- Journal of Neurochemistry
- Publisher
- Wiley
- Published
- 2016-10
- Volume
- 139
- Issue
- S2
- Pages
- 253-270
- ISSN
- 0022-3042 (ISSN-L)
- 1471-4159
- OpenAlex
- W2543701666
- Cited by
- 52
- References
- 14
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 1 · 2023: 1 · 2022: 4 · 2021: 2 · 2020: 16 · 2019: 9 · 2018: 7 · 2017: 5 · 2016: 1
- FWCI
- 1.54
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Neurochemistry (publishedVersion)
- PubMed (publishedVersion)
- JuSER (Forschungszentrum Jülich) (publishedVersion)
- RWTH Publications (RWTH Aachen) (submittedVersion)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Keywords
- research reproducibility, translational research, preclinical research, research misconduct, ARRIVE guidelines, Transparency and Openness Promotion, reporting guidelines, sexual dimorphism
- MeSH terms
- Animals, Clinical Trials as Topic, Humans, Statistics as Topic, Reproducibility of Results, Scientific Misconduct, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review