Geometric deep learning enables 3D kinematic profiling across species and environments
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Comprehensive descriptions of animal behavior require precise three-dimensional (3D) measurements of whole-body movements. Although two-dimensional approaches can track visible landmarks in restrictive environments, performance drops in freely moving animals, due to occlusions and appearance changes. Therefore, we designed DANNCE to robustly track anatomical landmarks in 3D across species and behaviors. DANNCE uses projective geometry to construct inputs to a convolutional neural network that leverages learned 3D geometric reasoning. We trained and benchmarked DANNCE using a dataset of nearly seven million frames that relates color videos and rodent 3D poses. In rats and mice, DANNCE robustly tracked dozens of landmarks on the head, trunk, and limbs of freely moving animals in naturalistic settings. We extended DANNCE to datasets from rat pups, marmosets, and chickadees, and demonstrate quantitative profiling of behavioral lineage during development. DANNCE enables robust 3D tracking of animals’ limbs and other features in naturalistic environments by making use of a deep learning approach that incorporates geometric reasoning. DANNCE is demonstrated on behavioral sequences from rodents, marmosets, and chickadees.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 1×
“These considerations have led to similar networks for 2D landmark detection in humans 57 .”
BackgroundDeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
On aims and methods of ethology
Tinbergen, N. (1963) · Zeitschrift für Tierpsychologie · doi:10.1111/j.1439-0310.1963.tb01161.x · Cited by 4,551
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Modulation of visual responses by behavioral state in mouse visual cortex
Niell, C. M., & Stryker, M. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.01.033 · Cited by 1,483 · cited 1×
“Armed with precise 3D kinematics of bodies and limbs in a diverse set of model systems, researchers will be able to refine their understanding of how a rich variety of movements, behaviors, and behavioral states are represented in, and generated by, different brain areas 47-50 .”
BackgroundNeurobiology of rodent self-grooming and its value for translational neuroscience
Kalueff, A. V., Stewart, A. M., Song, C., Berridge, K. C., Graybiel, A. M., & Fentress, J. C. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.8 · Cited by 852
Cited by
Anipose: A toolkit for robust markerless 3D pose estimation
Karashchuk, P., Rupp, K. L., Dickinson, E. S., Walling-Bell, S., Sanders, E., Azim, E., Brunton, B. W., & Tuthill, J. C. (2021) · Cell Reports · doi:10.1016/j.celrep.2021.109730 · Cited by 324
Identifying behavioral structure from deep variational embeddings of animal motion
Luxem, K., Mocellin, P., Fuhrmann, F., Kürsch, J., Miller, S. R., Palop, J. J., Remy, S., & Bauer, P. (2022) · Communications Biology · doi:10.1038/s42003-022-04080-7 · Cited by 218
Measuring behavior in the home cage: study design, applications, challenges, and perspectives
Grieco, F., Bernstein, B. J., Biemans, B., Bikovski, L., Burnett, C. J., Cushman, J. D., van Dam, E. A., Fry, S. A., Richmond-Hacham, B., Homberg, J. R., et al. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.735387 · Cited by 109
“Recent advances in deep learning for image processing have boosted the field to new heights, especially with the publication of DeepLabCut (Mathis et al., 2018), SLEAP (Pereira et al., 2019, 2020) and DANNCE (Dunn et al., 2021) for the estimation of body points.”
BackgroundOpen-source tools for behavioral video analysis: Setup, methods, and best practices
Luxem, K., Sun, J. J., Bradley, S. P., Krishnan, K., Yttri, E., Zimmermann, J., Pereira, T. D., & Laubach, M. (2023) · eLife · doi:10.7554/elife.79305 · Cited by 98
A virtual rodent predicts the structure of neural activity across behaviours
Aldarondo, D., Merel, J., Marshall, J. D., Hasenclever, L., Klibaite, U., Gellis, A., Tassa, Y., Wayne, G., Botvinick, M., & Ölveczky, B. P. (2024) · Nature · doi:10.1038/s41586-024-07633-4 · Cited by 70
Self-citation
Metadata
- Authors
- Dunn, T. W., Marshall, J. D., Severson, K. S., Aldarondo, D. E., Hildebrand, D. G. C., Chettih, S. N., Wang, W. L., Gellis, A. J., Carlson, D. E., Aronov, D., Freiwald, W. A., Wang, F., & Ölveczky, B. P.
- Venue
- Nature Methods
- Published
- 2021-04-19
- Volume
- 18
- Issue
- 5
- Pages
- 564-573
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- OpenAlex
- W3156592713
- Cited by
- 211
- References
- 20
- FWCI
- 11.90
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Geometric deep learning enables 3D kinematic profiling across species and environments. https://app.cochat.ai/papers/10.1038/s41592-021-01106-6
- The paper
- Dunn, T. W., Marshall, J. D., Severson, K. S., Aldarondo, D. E., Hildebrand, D. G. C., Chettih, S. N., Wang, W. L., Gellis, A. J., Carlson, D. E., Aronov, D., Freiwald, W. A., Wang, F., & Ölveczky, B. P. (2021). Geometric deep learning enables 3D kinematic profiling across species and environments. Nature Methods. https://doi.org/10.1038/s41592-021-01106-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.