Cited by 215Animal testing and alternativesResearch Data Management PracticesPhilosophy and History of Science

Reproducibility and replicability of rodent phenotyping in preclinical studies

Open access
Kafkafi, Neri, Agassi, Joseph, Chesler, Elissa J., Crabbe, John C., Crusio, Wim E., Eilam, David, Gerlai, Robert, Golani, Ilan +18 more·2018·Neuroscience & Biobehavioral Reviews

Published version of a preprint

Abstract

The scientific community is increasingly concerned with the proportion of published "discoveries" that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.neubiorev.2018.01.003

Cites

195 of 195 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Animals, experimental units, or data points that are unaccounted for can lead to instances in which conclusions cannot be supported by the raw data.

    “Animals, experimental units, or data points that are unaccounted for can lead to instances in which conclusions cannot be supported by the raw data [52].”

    From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [52]

    The section “Reproducibility and replicability in general science: a crisis?” (page unavailable) says that some studies are difficult to reproduce because their figures, tables, and conclusions cannot be regenerated from the study data. It does not identify unaccounted-for animals, experimental units, or data points as the cause, so that specific part of the claim is unsupported. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

1 retracted

Flagged first, then by citations

Research works printed but not linked: 6

  1. Anderson PW, 1972. More is different. Science 177 (4047), 393– 396. 17796623
  2. Beynen AC, Gärtner K, van Zutphen LFM, 2003. In: van Zutphen LFM, Baumans V, Beynen AC (Eds.), Standardization of Animal Experimentation In Principles of Laboratory Animal Science. Elsevier, Amsterdam, pp. 103– 110.
  3. Chesler EJ, Wang J, Lu L, Qu Y, Manly KF, Williams RW, 2003. Genetic correlates of gene expression in recombinant inbred strains. Neuroinformatics 1 (4), 343– 357. 15043220
  4. Diggle PJ, Zeger SL, 2010. Embracing the concept of reproducible research. Biostatistics 11 (3), 375. 20538869
  5. Stark PB, 2017. Nullius in verba. In: Kitzes J, Turek D, Deniz F (Eds.), The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences. University of California Press, Oakland, CA, pp. 6– 9. (Accessed 8 June, 2017). https://www.practicereproducibleresearch.org/core-chapters/0-preface.html.
  6. Stodden V, Guo P, Ma Z, 2013. Toward reproducible computational research: an empirical analysis of data and code policy adoption by journals. PLoS One 8 (June (6)), e67111. 10.1371/journal.pone.0067111. 23805293 PMC3689732 · DOI · Paper page

Cited by

Sorted by citations

Authors
Kafkafi, N., Agassi, J., Chesler, E. J., Crabbe, J. C., Crusio, W. E., Eilam, D., Gerlai, R., Golani, I., Gomez-Marin, A., Heller, R., Iraqi, F., Jaljuli, I., Karp, N. A., Morgan, H., Nicholson, G., Pfaff, D. W., Richter, S. H., Stark, P. B., Stiedl, O., Stodden, V., Tarantino, L. M., Tucci, V., Valdar, W., Williams, R. W., Würbel, H., & Benjamini, Y.
Affiliations
  • Tel Aviv University (IL)
  • Jackson Laboratory (US)
  • Oregon Health & Science University (US)
  • Centre National de la Recherche Scientifique (FR)
  • University of Toronto (CA)
  • AstraZeneca (United Kingdom) (GB)
  • University of Oxford (GB)
  • Rockefeller University (US)
  • University of Münster (DE)
  • University of California, Berkeley (US)
  • Vrije Universiteit Amsterdam (NL)
  • University of Illinois Urbana-Champaign (US)
  • University of North Carolina at Chapel Hill (US)
  • Italian Institute of Technology (IT)
  • University of Tennessee Health Science Center (US)
  • University of Bern (CH)
Corresponding author
Kafkafi, Neri
Venue
Neuroscience & Biobehavioral Reviews
Publisher
Elsevier BV
Published
2018-04
Volume
87
Pages
218-232
ISSN
0149-7634 (ISSN-L)
OpenAlex
W2952728789
Cited by
215
References
194
Citations by year
2026: 20 · 2025: 23 · 2024: 26 · 2023: 25 · 2022: 26 · 2021: 25 · 2020: 37 · 2019: 21 · 2018: 9 · 2017: 2
FWCI
4.86
Type
journal article
Language
en
Open access
Locations
  • Neuroscience & Biobehavioral Reviews (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • VU Research Portal (publishedVersion, other-oa)
  • Europe PMC (PubMed Central) (acceptedVersion, other-oa)
  • PubMed Central (acceptedVersion, cc-by-nc-nd)
  • HAL (submittedVersion)
  • DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion, cc-by-nc-nd)
  • eScholarship (California Digital Library) (submittedVersion, cc-by-nc-nd)
  • The Mouseion at the JAXlibrary (Jackson Laboratory) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Funders
Spanish Ministry of Economy and Competitivity (Severo Ochoa Center of Excellence programme
Topics
  • Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
  • Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
Keywords
reproducibility, replicability, preclinical research, gene-environment interaction, research ethics, translational research
MeSH terms
Animals, Behavior, Animal, Phenotype, Research, Research Design, Rodentia, Reproducibility of Results, Models, Animal, Animal Experimentation, Information Dissemination

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review