Continuous Whole-Body 3D Kinematic Recordings across the Rodent Behavioral Repertoire
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
In mammalian animal models, high-resolution kinematic tracking is restricted to brief sessions in constrained environments, limiting our ability to probe naturalistic behaviors and their neural underpinnings. To address this, we developed CAPTURE, a behavioral monitoring system that combines motion capture and deep learning to continuously track the 3D kinematics of a rat’s head, trunk, and limbs for week-long timescales in freely behaving animals. CAPTURE realizes 10–100 fold gains in precision and robustness compared with existing convolutional network approaches to behavioral tracking. We demonstrate CAPTURE’s ability to comprehensively profile the kinematics and sequential organization of natural rodent behavior, its variation across individuals, and its perturbation by drugs and disease, including identifying perseverative grooming states in a rat model of Fragile X syndrome. CAPTURE significantly expands the range of behaviors and contexts that can be quantitatively investigated, opening the door to a new understanding of natural behavior and its neural basis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187 · cited 4×
“Recently, 2D convolutional networks trained from scratch or on human-labeled pose databases have been applied to detect un-occluded visual landmarks, mostly in 2D, in individual behavioral tasks (Mathis et al., 2018; Pereira et al., 2019).”
BackgroundInfluentialWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 2×
“Extending CAPTURE to a broader range of species and keypoint locations could be helped by using it alongside synchronized video recordings, to train deep learning algorithms for markerless 3D motion capture as is done in humans (Ionescu et al., 2014; Pavlakos et al., 2017).”
BackgroundNeural consequences of enviromental enrichment
van Praag, H., Kempermann, G., & Gage, F. H. (2000) · Nature Reviews Neuroscience · doi:10.1038/35044558 · Cited by 2,468
Natural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · cited 1×
“Similar to how naturalistic visual stimuli were essential for illuminating the function and organization of the visual system (Simoncelli and Olshausen, 2001), effective tools for measuring naturalistic behavior should enable a new understanding of the principles by which the motor system stores and produces behavior…”
BackgroundAnimal models of neuropsychiatric disorders
Nestler, E. J., & Hyman, S. E. (2010) · Nature Neuroscience · doi:10.1038/nn.2647 · Cited by 2,023 · cited 1×
“The ability of CAPTURE to record natural behaviors over long timescales should be especially useful in models of behavioral disorders such as autism, anxiety, bipolar, depression, in which symptoms are often associated with slower changes in behavioral patterns that have been difficult to reliably measure in animal mo…”
BackgroundBehavioural phenotyping assays for mouse models of autism
Silverman, J. L., Yang, M., Lord, C., & Crawley, J. N. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2851 · Cited by 1,462 · cited 3×
“More generally, they could serve as biomarkers for various disease models, the utility of which are currently limited by the lack of behavioral reproducibility across laboratories (Brunner et al., 2015; Silverman et al., 2010).”
BackgroundNeuroscience Needs Behavior: Correcting a Reductionist Bias
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A., & Poeppel, D. (2017) · Neuron · doi:10.1016/j.neuron.2016.12.041 · Cited by 1,358 · cited 1×
“A critical first step towards this goal is the development of tools and analysis frameworks capable of precisely measuring and describing the behavior of our experimental subjects (Anderson and Perona, 2014; Egnor and Branson, 2016; Krakauer et al., 2017).”
BackgroundNeurobiology of rodent self-grooming and its value for translational neuroscience
Kalueff, A. V., Stewart, A. M., Song, C., Berridge, K. C., Graybiel, A. M., & Fentress, J. C. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.8 · Cited by 852
Mapping Sub-Second Structure in Mouse Behavior
Wiltschko, A. B., Johnson, M. J., Iurilli, G., Peterson, R. E., Katon, J. M., Pashkovski, S. L., Abraira, V. E., Adams, R. P., & Datta, S. R. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.031 · Cited by 760 · cited 6×
“, 2010) or depth cameras to coarsely assess changes in usage across a broader range of natural behaviors (Wiltschko et al., 2015), CAPTURE can comprehensively identify precise changes in both behavioral kinematics and usage.”
BackgroundMethodInfluentialA framework for studying emotions across species
Anderson, D. J., & Adolphs, R. (2014) · Cell · doi:10.1016/j.cell.2014.03.003 · Cited by 691 · cited 1×
“However, such unsupervised analyses will only be an intermediate step towards the creation of standardized and rigorously defined behavioral taxonomies, which CAPTURE recordings should greatly facilitate (Anderson and Adolphs, 2014).”
BackgroundJAABA: Interactive machine learning for automatic annotation of animal behavior
Kabra, M., Robie, A. A., Rivera-Alba, M., Branson, S., & Branson, K. (2013) · Nature Methods · doi:10.1038/nmeth.2281 · Cited by 644
Mapping the stereotyped behaviour of freely moving fruit flies
Berman, G. J., Choi, D. M., Bialek, W., & Shaevitz, J. W. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0672 · Cited by 565
Self-citationEmergence of Individuality in Genetically Identical Mice
Freund, J., Brandmaier, A. M., Lewejohann, L., Kirste, I., Kritzler, M., Krüger, A., Sachser, N., Lindenberger, U., & Kempermann, G. (2013) · Science · doi:10.1126/science.1235294 · Cited by 536 · cited 1×
“Individuality is typically measured through coarse estimates of behavioral usage (Forkosh et al., 2019; Freund et al., 2013), leaving open the question of whether animals exhibit individuality along other dimensions, for instance in the kinematics and long-timescale patterning of behavior that can now be systematicall…”
BackgroundDimensionality and Dynamics in the Behavior of C. elegans
Stephens, G. J., Johnson-Kerner, B., Bialek, W., & Ryu, W. S. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000028 · Cited by 509 · cited 2×
“Behavioral Feature Generation and Selection—To detect specific behaviors and kinematic variants of behaviors in our datasets, we generated a set of 140 per-frame features that describe the instantaneous speed and spatial configuration of the motion capture markers, as well as the spatio-temporal trajectories of the ma…”
BackgroundMethodNeural mechanisms of movement planning: motor cortex and beyond
Svoboda, K., & Li, N. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.023 · Cited by 347 · cited 1×
“However, while task-based studies have yielded detailed characterizations of how individual movements are produced (Dhawale et al., 2019; Svoboda and Li, 2018), how more complex whole-body behaviors are controlled and generated remain poorly understood.”
BackgroundAutomated long-term recording and analysis of neural activity in behaving animals
Dhawale, A. K., Poddar, R., Wolff, S. B., Normand, V. A., Kopelowitz, E., & Ölveczky, B. P. (2017) · eLife · doi:10.7554/elife.27702 · Cited by 159
Self-citationAntoniou, K., Kafetzopoulos, E., Papadopoulou-Daifoti, Z., Hyphantis, T., & Marselos, M. (1998) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(98)00020-7 · Cited by 127 · cited 2×
“In the past, amphetamine was thought to elicit a biphasic behavioral response, stimulating locomotor activity at low doses and repetitive motor ‘stereotypies’, such as head swinging, at higher doses (Antoniou et al., 1998).”
BackgroundMethodEfficient cortical coding of 3D posture in freely behaving rats
Mimica, B., Dunn, B. A., Tombaz, T., Bojja, V. P. T. N. C. S., & Whitlock, J. R. (2018) · Science · doi:10.1126/science.aau2013 · Cited by 124
Brunner, D., Kabitzke, P., He, D., Cox, K., Thiede, L., Hanania, T., Sabath, E., Alexandrov, V., Saxe, M., Peles, E., et al. (2015) · PLOS ONE · doi:10.1371/journal.pone.0134572 · Cited by 111
Identity domains capture individual differences from across the behavioral repertoire
Forkosh, O., Karamihalev, S., Roeh, S., Alon, U., Anpilov, S., Touma, C., Nussbaumer, M., Flachskamm, C., Kaplick, P. M., Shemesh, Y., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0516-y · Cited by 110
The manifold structure of limb coordination in walking Drosophila
DeAngelis, B. D., Zavatone-Veth, J. A., & Clark, D. A. (2019) · eLife · doi:10.7554/elife.46409 · Cited by 104 · cited 2×
“…to readout postural kinematics also opens the possibility of creating analysis frameworks that go beyond behavioral identification to elucidate important axes of variation from the perspective of motor control (DeAngelis et al., 2019), behavioral exploration (Wu et al., 2014), and neural activity.”
BackgroundDeep neuroethology of a virtual rodent
Merel, J., Aldarondo, D., Marshall, J., Tassa, Y., Wayne, G., & Ölveczky, B. (2019) · arXiv · doi:10.48550/arxiv.1911.09451 · Cited by 44
Self-citation
Cited by
Anipose: A toolkit for robust markerless 3D pose estimation
Karashchuk, P., Rupp, K. L., Dickinson, E. S., Walling-Bell, S., Sanders, E., Azim, E., Brunton, B. W., & Tuthill, J. C. (2021) · Cell Reports · doi:10.1016/j.celrep.2021.109730 · Cited by 324
Catalyzing next-generation Artificial Intelligence through NeuroAI
Zador, A., Escola, S., Richards, B., Ölveczky, B., Bengio, Y., Boahen, K., Botvinick, M., Chklovskii, D., Churchland, A., Clopath, C., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-37180-x · Cited by 300
Self-citationKeypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics
Weinreb, C., Pearl, J. E., Lin, S., Osman, M. A. M., Zhang, L., Annapragada, S., Conlin, E., Hoffmann, R., Makowska, S., Gillis, W. F., et al. (2024) · Nature Methods · doi:10.1038/s41592-024-02318-2 · Cited by 228
A virtual rodent predicts the structure of neural activity across behaviours
Aldarondo, D., Merel, J., Marshall, J. D., Hasenclever, L., Klibaite, U., Gellis, A., Tassa, Y., Wayne, G., Botvinick, M., & Ölveczky, B. P. (2024) · Nature · doi:10.1038/s41586-024-07633-4 · Cited by 70
Self-citationBehavioral decomposition reveals rich encoding structure employed across neocortex in rats
Mimica, B., Tombaz, T., Battistin, C., Fuglstad, J. G., Dunn, B. A., & Whitlock, J. R. (2023) · Nature Communications · doi:10.1038/s41467-023-39520-3 · Cited by 45
Metadata
- Authors
- Marshall, J. D., Aldarondo, D. E., Dunn, T. W., Wang, W. L., Berman, G. J., & Ölveczky, B. P.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2021-02
- Volume
- 109
- Issue
- 3
- Pages
- 420-437.e8
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W3111687703
- Cited by
- 157
- References
- 29
- FWCI
- 8.86
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.