Standardized and reproducible measurement of decision-making in mice
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The UK Biobank resource with deep phenotyping and genomic data
Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L. T., Sharp, K., Motyer, A., Vukcevic, D., Delaneau, O., O’Connell, J., et al. (2018) · Nature · doi:10.1038/s41586-018-0579-z · Cited by 8,876
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
seaborn: statistical data visualization
Waskom, M. (2021) · Journal of Open Source Software · doi:10.21105/joss.03021 · Cited by 7,310
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
Genetics of Mouse Behavior: Interactions with Laboratory Environment
Crabbe, J. C., Wahlsten, D., & Dudek, B. C. (1999) · Science · doi:10.1126/science.284.5420.1670 · Cited by 1,666
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521
What does research reproducibility mean?
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293
High-throughput discovery of novel developmental phenotypes
The International Mouse Phenotyping Consortium, Dickinson, M. E., Flenniken, A. M., Ji, X., Teboul, L., Wong, M. D., White, J. K., Meehan, T. F., Weninger, W. J., Westerberg, H., et al. (2016) · Nature · doi:10.1038/nature19356 · Cited by 1,196
Olfactory exposure to males, including men, causes stress and related analgesia in rodents
Sorge, R. E., Martin, L. J., Isbester, K. A., Sotocinal, S. G., Rosen, S., Tuttle, A. H., Wieskopf, J. S., Acland, E. L., Dokova, A., Kadoura, B., et al. (2014) · Nature Methods · doi:10.1038/nmeth.2935 · Cited by 905
Waking state: rapid variations modulate neural and behavioral responses
McGinley, M. J., Vinck, M., Reimer, J., Batista-Brito, R., Zagha, E., Cadwell, C. R., Tolias, A. S., Cardin, J. A., & McCormick, D. A. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.012 · Cited by 801
Replications in Psychology Research: How Often Do They Really Occur?
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460688 · Cited by 750
Beyond authorship: attribution, contribution, collaboration, and credit
Brand, A., Allen, L., Altman, M., Hlava, M., & Scott, J. (2015) · Learned Publishing · doi:10.1087/20150211 · Cited by 678
Deep Multilayer Brain Proteomics Identifies Molecular Networks in Alzheimer’s Disease Progression
Bai, B., Wang, X., Li, Y., Chen, P.-C., Yu, K., Dey, K. K., Yarbro, J. M., Han, X., Lutz, B. M., Rao, S., et al. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.015 · Cited by 565
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485
Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias
Urai, A. E., Braun, A., & Donner, T. H. (2017) · Nature Communications · doi:10.1038/ncomms14637 · Cited by 466
Self-citationElapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task
Hanks, T. D., Mazurek, M. E., Kiani, R., Hopp, E., & Shadlen, M. N. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.5613-10.2011 · Cited by 378
Mice alternate between discrete strategies during perceptual decision-making
Ashwood, Z. C., Roy, N. A., Stone, I. R., Laboratory, T. I. B., Urai, A. E., Churchland, A. K., Pouget, A., & Pillow, J. W. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-01007-z · Cited by 229
Self-citationReproducibility and replicability of rodent phenotyping in preclinical studies
Kafkafi, N., Agassi, J., Chesler, E. J., Crabbe, J. C., Crusio, W. E., Eilam, D., Gerlai, R., Golani, I., Gomez-Marin, A., Heller, R., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.01.003 · Cited by 215
Citric Acid Water as an Alternative to Water Restriction for High-Yield Mouse Behavior
Urai, A. E., Aguillon-Rodriguez, V., Laranjeira, I. C., Cazettes, F., Laboratory, T. I. B., Mainen, Z. F., & Churchland, A. K. (2020) · doi:10.1101/2020.03.02.973016 · Cited by 12
Self-citation
Cited by
Large-scale neural recordings call for new insights to link brain and behavior
Urai, A. E., Doiron, B., Leifer, A. M., & Churchland, A. K. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-00980-9 · Cited by 328
Self-citationA standardized framework to test event-based experiments
Lepauvre, A., Hirschhorn, R., Bendtz, K., Mudrik, L., & Melloni, L. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02508-y · Cited by 9
Metadata
- Authors
- Laboratory, T. I. B., Aguillon-Rodriguez, V., Angelaki, D., Bayer, H., Bonacchi, N., Carandini, M., Cazettes, F., Chapuis, G., Churchland, A. K., Dan, Y., Dewitt, E., Faulkner, M., Forrest, H., Haetzel, L., Häusser, M., Hofer, S. B., Hu, F., Khanal, A., Krasniak, C., Laranjeira, I., Mainen, Z. F., Meijer, G., Miska, N. J., Mrsic-Flogel, T. D., Murakami, M., Noel, J.-P., Pan-Vazquez, A., Rossant, C., Sanders, J., Socha, K., Terry, R., Urai, A. E., Vergara, H., Wells, M., Wilson, C. J., Witten, I. B., Wool, L. E., & Zador, A. M.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2021-05-20
- Volume
- 10
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W3162611316
- Cited by
- 189
- References
- 22
- FWCI
- 12.93
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.