The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
SLEAP: A deep learning system for multi-animal pose tracking
Open accessAbstract
The desire to understand how the brain generates and patterns behavior has driven rapid methodological innovation in tools to quantify natural animal behavior. While advances in deep learning and computer vision have enabled markerless pose estimation in individual animals, extending these to multiple animals presents unique challenges for studies of social behaviors or animals in their natural environments. Here we present Social LEAP Estimates Animal Poses (SLEAP), a machine learning system for multi-animal pose tracking. This system enables versatile workflows for data labeling, model training and inference on previously unseen data. SLEAP features an accessible graphical user interface, a standardized data model, a reproducible configuration system, over 30 model architectures, two approaches to part grouping and two approaches to identity tracking. We applied SLEAP to seven datasets across flies, bees, mice and gerbils to systematically evaluate each approach and architecture, and we compare it with other existing approaches. SLEAP achieves greater accuracy and speeds of more than 800 frames per second, with latencies of less than 3.5 ms at full 1,024 × 1,024 image resolution. This makes SLEAP usable for real-time applications, which we demonstrate by controlling the behavior of one animal on the basis of the tracking and detection of social interactions with another animal.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423 · cited 1×
“In particular, we designed our data model with the goal of mitigating the risk of data dependencies in complex machine learning systems 27 as well as to promote reproducibility and FAIR scientific data-stewardship best practices 30 .”
MethodObservational study of behavior: sampling methods
Altmann, J. (1974) · Behaviour · doi:10.1163/156853974x00534 · Cited by 13,891
Reducibility among Combinatorial Problems
Karp, R. M. (1972) · Complexity of Computer Computations · doi:10.1007/978-1-4684-2001-2_9 · Cited by 11,511
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 17×
“To generate PAFs from labeled data, the user must define a directed graph, which we refer to as the skeleton, that connects all body parts to be tracked.”
BackgroundMethodInfluential
Research works printed but not linked: 5
- Vogt, N. Automated behavioral analysis. Nat. Methods 18, 29 (2021). · doi:10.1038/s41592-020-01030-1
- Pereira, T. D. et al. SLEAP Datasets and Models https://doi.org/10.17605/OSF.IO/36HAR (2021). · doi:10.17605/osf.io/36har
- Murray, D. G., Simsa, J., Klimovic, A. & Indyk, I. tf.data: a machine learning data processing framework. Preprint at https://arxiv.org/abs/2101.12127 (2021). · arXiv
- Nibali, A., He, Z., Morgan, S. & Prendergast, L. Numerical coordinate regression with convolutional neural networks. Preprint at https://arxiv.org/abs/1801.07372 (2018). · arXiv
- Moldovan, D. et al. AutoGraph: imperative-style coding with graph-based performance. Preprint at https://arxiv.org/abs/1810.08061 (2018). · arXiv
Cited by
Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics
Weinreb, C., Pearl, J. E., Lin, S., Osman, M. A. M., Zhang, L., Annapragada, S., Conlin, E., Hoffmann, R., Makowska, S., Gillis, W. F., et al. (2024) · Nature Methods · doi:10.1038/s41592-024-02318-2 · Cited by 228
Self-citationIdentifying behavioral structure from deep variational embeddings of animal motion
Luxem, K., Mocellin, P., Fuhrmann, F., Kürsch, J., Miller, S. R., Palop, J. J., Remy, S., & Bauer, P. (2022) · Communications Biology · doi:10.1038/s42003-022-04080-7 · Cited by 218
Open-source tools for behavioral video analysis: Setup, methods, and best practices
Luxem, K., Sun, J. J., Bradley, S. P., Krishnan, K., Yttri, E., Zimmermann, J., Pereira, T. D., & Laubach, M. (2023) · eLife · doi:10.7554/elife.79305 · Cited by 98
Self-citationNeural circuits regulating prosocial behaviors
Walsh, J. J., Christoffel, D. J., & Malenka, R. C. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-022-01348-8 · Cited by 72
Mapping model units to visual neurons reveals population code for social behaviour
Cowley, B. R., Calhoun, A. J., Rangarajan, N., Ireland, E., Turner, M. H., Pillow, J. W., & Murthy, M. (2024) · Nature · doi:10.1038/s41586-024-07451-8 · Cited by 55
“In total, behaviour was recorded and analysed from 459 pairs; the number of flies per condition were as follows: Joint positions for the male and female for every frame were tracked with a DNN trained for multi-animal pose estimation called SLEAP 51 .”
MethodSelf-citation
Metadata
- Authors
- Pereira, T. D., Tabris, N., Matsliah, A., Turner, D. M., Li, J., Ravindranath, S., Papadoyannis, E. S., Normand, E., Deutsch, D. S., Wang, Z. Y., McKenzie-Smith, G. C., Mitelut, C. C., Castro, M. D., D’Uva, J., Kislin, M., Sanes, D. H., Kocher, S. D., Wang, S. S.-H., Falkner, A. L., Shaevitz, J. W., & Murthy, M.
- Venue
- Nature Methods
- Published
- 2022-04
- Volume
- 19
- Issue
- 4
- Pages
- 486-495
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- OpenAlex
- W4223962547
- Cited by
- 886
- References
- 38
- FWCI
- 53.66
- Type
- journal article
- Language
- en
- Update notices
- correction (2022-04-25)
- Open access
- hybrid
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor, 3 d)
- Funders
- National Science Foundation (PHY-1734030, DGE-1148900)
- U.S. Department of Health & Human Services | National Institutes of Health (R35 NS111580-02, R01 NS104899, R01 DC011284, GM137424-01, R00 MH109674)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SLEAP: A deep learning system for multi-animal pose tracking. https://app.cochat.ai/papers/10.1038/s41592-022-01426-1
- The paper
- Pereira, T. D., Tabris, N., Matsliah, A., Turner, D. M., Li, J., Ravindranath, S., Papadoyannis, E. S., Normand, E., Deutsch, D. S., Wang, Z. Y., McKenzie-Smith, G. C., Mitelut, C. C., Castro, M. D., D’Uva, J., Kislin, M., Sanes, D. H., Kocher, S. D., Wang, S. S.-H., Falkner, A. L., . . . Murthy, M. (2022). SLEAP: A deep learning system for multi-animal pose tracking. Nature Methods. https://doi.org/10.1038/s41592-022-01426-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
