The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
TRex, a fast multi-animal tracking system with markerless identification, and 2D estimation of posture and visual fields
Open accessAbstract
Automated visual tracking of animals is rapidly becoming an indispensable tool for the study of behavior. It offers a quantitative methodology by which organisms’ sensing and decision-making can be studied in a wide range of ecological contexts. Despite this, existing solutions tend to be challenging to deploy in practice, especially when considering long and/or high-resolution video-streams. Here, we present TRex, a fast and easy-to-use solution for tracking a large number of individuals simultaneously using background-subtraction with real-time (60 Hz) tracking performance for up to approximately 256 individuals and estimates 2D visual-fields, outlines, and head/rear of bilateral animals, both in open and closed-loop contexts. Additionally, TRex offers highly accurate, deep-learning-based visual identification of up to approximately 100 unmarked individuals, where it is between 2.5 and 46.7 times faster, and requires 2–10 times less memory, than comparable software (with relative performance increasing for more organisms/longer videos) and provides interactive data-exploration within an intuitive, platform-independent graphical user-interface.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
(2010) · Cited by 14,134
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556
Visual pattern recognition by moment invariants
Hu, M.-K. (1962) · IEEE Transactions on Information Theory · doi:10.1109/tit.1962.1057692 · Cited by 7,546
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187
Receptive fields of single neurones in the cat's striate cortex
Hubel, D. H., & Wiesel, T. N. (1959) · The Journal of Physiology · doi:10.1113/jphysiol.1959.sp006308 · Cited by 4,653
Algorithms for the Assignment and Transportation Problems
Munkres, J. (1957) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0105003 · Cited by 3,986
Fibonacci heaps and their uses in improved network optimization algorithms
Fredman, M. L., & Tarjan, R. E. (1987) · Journal of the ACM · doi:10.1145/28869.28874 · Cited by 3,132
The Effectiveness of Data Augmentation in Image Classification using Deep Learning
Perez, L., & Wang, J. (2017) · arXiv · doi:10.48550/arxiv.1712.04621 · Cited by 2,499
An Automatic Method for Solving Discrete Programming Problems
Land, A. H., & Doig, A. G. (2010) · 50 Years of Integer Programming 1958-2008 · doi:10.1007/978-3-540-68279-0_5 · Cited by 1,988
Elliptic Fourier features of a closed contour
Kuhl, F. P., & Giardina, C. R. (1982) · Computer Graphics and Image Processing · doi:10.1016/0146-664x(82)90034-x · Cited by 1,839
Spatial and chromatic interactions in the lateral geniculate body of the rhesus monkey.
Wiesel, T. N., & Hubel, D. H. (1966) · Journal of Neurophysiology · doi:10.1152/jn.1966.29.6.1115 · Cited by 1,275
RECEPTIVE FIELDS OF CELLS IN STRIATE CORTEX OF VERY YOUNG, VISUALLY INEXPERIENCED KITTENS
Hubel, D. H., & Wiesel, T. N. (1963) · Journal of Neurophysiology · doi:10.1152/jn.1963.26.6.994 · Cited by 1,116
Neocognitron: A hierarchical neural network capable of visual pattern recognition
Fukushima, K. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90014-7 · Cited by 1,097
An Algorithm for the Traveling Salesman Problem
Little, J. D. C., Murty, K. G., Sweeney, D. W., & Karel, C. (1963) · Operations Research · doi:10.1287/opre.11.6.972 · Cited by 1,058
One-Shot Video Object Segmentation
Caelles, S., Maninis, K.-K., Pont-Tuset, J., Leal-Taixe, L., Cremers, D., & Van Gool, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.565 · Cited by 927
High-throughput ethomics in large groups of Drosophila
Branson, K., Robie, A. A., Bender, J., Perona, P., & Dickinson, M. H. (2009) · Nature Methods · doi:10.1038/nmeth.1328 · Cited by 786
idTracker: tracking individuals in a group by automatic identification of unmarked animals
Pérez-Escudero, A., Vicente-Page, J., Hinz, R. C., Arganda, S., & de Polavieja, G. G. (2014) · Nature Methods · doi:10.1038/nmeth.2994 · Cited by 774
EthoVision: A versatile video tracking system for automation of behavioral experiments
Noldus, L. P. J. J., Spink, A. J., & Tegelenbosch, R. A. J. (2001) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195394 · Cited by 717
Fast animal pose estimation using deep neural networks
Pereira, T. D., Aldarondo, D. E., Willmore, L., Kislin, M., Wang, S. S.-H., Murthy, M., & Shaevitz, J. W. (2019) · Nature Methods · doi:10.1038/s41592-018-0234-5 · Cited by 687
Colavita, F. B. (1974) · Perception & Psychophysics · doi:10.3758/bf03203962 · Cited by 559
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Graving, J. M., Chae, D., Naik, H., Li, L., Koger, B., Costelloe, B. R., & Couzin, I. D. (2019) · eLife · doi:10.7554/elife.47994 · Cited by 547
Self-citationAutomated image-based tracking and its application in ecology
Dell, A. I., Bender, J. A., Branson, K., Couzin, I. D., de Polavieja, G. G., Noldus, L. P. J. J., Pérez-Escudero, A., Perona, P., Straw, A. D., Wikelski, M., et al. (2014) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2014.05.004 · Cited by 492
Generation of fiducial marker dictionaries using Mixed Integer Linear Programming
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F. J., & Medina-Carnicer, R. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.09.023 · Cited by 490
Rosenthal, S. B., Twomey, C. R., Hartnett, A. T., Wu, H. S., & Couzin, I. D. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1420068112 · Cited by 474
Self-citationToxTrac: A fast and robust software for tracking organisms
Rodriguez, A., Zhang, H., Klaminder, J., Brodin, T., Andersson, P. L., & Andersson, M. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12874 · Cited by 465
Visual sensory networks and effective information transfer in animal groups
Strandburg-Peshkin, A., Twomey, C. R., Bode, N. W. F., Kao, A. B., Katz, Y., Ioannou, C. C., Rosenthal, S. B., Torney, C. J., Wu, H. S., Levin, S. A., et al. (2013) · Current Biology · doi:10.1016/j.cub.2013.07.059 · Cited by 428
Self-citationTracking Individuals Shows Spatial Fidelity Is a Key Regulator of Ant Social Organization
Mersch, D. P., Crespi, A., & Keller, L. (2013) · Science · doi:10.1126/science.1234316 · Cited by 418
Linear-time connected-component labeling based on sequential local operations
Suzuki, K., Horiba, I., & Sugie, N. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(02)00030-9 · Cited by 414
idtracker.ai: tracking all individuals in small or large collectives of unmarked animals
Romero-Ferrero, F., Bergomi, M. G., Hinz, R. C., Heras, F. J. H., & de Polavieja, G. G. (2019) · Nature Methods · doi:10.1038/s41592-018-0295-5 · Cited by 406
Video Object Segmentation without Temporal Information
Maninis, K.-K., Caelles, S., Chen, Y., Pont-Tuset, J., Leal-Taixe, L., Cremers, D., & Van Gool, L. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2838670 · Cited by 354
Fast connected-component labeling
He, L., Chao, Y., Suzuki, K., & Wu, K. J. (2008) · Pattern Recognition · doi:10.1016/j.patcog.2008.10.013 · Cited by 342
Subdivision Methods for Geometric Design: A Constructive Approach
Warren, J. D., & Weimer, H. (2001) · Cited by 292
A new algorithm for the assignment problem
Bertsekas, D. P. (1981) · Mathematical Programming · doi:10.1007/bf01584237 · Cited by 292
Challenges and solutions for studying collective animal behaviour in the wild
Hughey, L. F., Hein, A. M., Strandburg-Peshkin, A., & Jensen, F. H. (2018) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2017.0005 · Cited by 228
Visuomotor Transformations Underlying Hunting Behavior in Zebrafish
Bianco, I. H., & Engert, F. (2015) · Current Biology · doi:10.1016/j.cub.2015.01.042 · Cited by 217
The zebrafish as a model visual system
Bilotta, J., & Saszik, S. (2001) · International Journal of Developmental Neuroscience · doi:10.1016/s0736-5748(01)00050-8 · Cited by 187
Social behavior recognition in continuous video
Burgos-Artizzu, X. P., Dollár, P., Lin, D., Anderson, D. J., & Perona, P. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247817 · Cited by 180
Context-dependent hierarchies in pigeons
Nagy, M., Vásárhelyi, G., Pettit, B., Roberts-Mariani, I., Vicsek, T., & Biro, D. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1305552110 · Cited by 178
BEEtag: A Low-Cost, Image-Based Tracking System for the Study of Animal Behavior and Locomotion
Crall, J. D., Gravish, N., Mountcastle, A. M., & Combes, S. A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0136487 · Cited by 166
Applications of radio frequency identification (RFID) in ornithological research: a review
Bonter, D. N., & Bridge, E. S. (2011) · Journal of Field Ornithology · doi:10.1111/j.1557-9263.2010.00302.x · Cited by 165
Sockman, K. W., & Schwabl, H. (2001) · General and Comparative Endocrinology · doi:10.1006/gcen.2001.7626 · Cited by 165
Pankiw, T., & Page, R. E. (2003) · Journal of Comparative Physiology A · doi:10.1007/s00359-003-0442-y · Cited by 159
Automated multi-day tracking of marked mice for the analysis of social behaviour
Ohayon, S., Avni, O., Taylor, A. L., Perona, P., & Egnor, S. E. R. (2013) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2013.05.013 · Cited by 156
FlyMAD: rapid thermogenetic control of neuronal activity in freely walking Drosophila
Bath, D. E., Stowers, J. R., Hörmann, D., Poehlmann, A., Dickson, B. J., & Straw, A. D. (2014) · Nature Methods · doi:10.1038/nmeth.2973 · Cited by 152
Order and Flexibility in the Motion of Fish Schools
Inada, Y., & Kawachi, K. (2002) · Journal of Theoretical Biology · doi:10.1006/jtbi.2001.2449 · Cited by 147
Tracktor: Image‐based automated tracking of animal movement and behaviour
Sridhar, V. H., Roche, D. G., & Gingins, S. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13166 · Cited by 141
Machine vision methods for analyzing social interactions
Robie, A. A., Seagraves, K. M., Egnor, S. E. R., & Branson, K. M. (2017) · Journal of Experimental Biology · doi:10.1242/jeb.142281 · Cited by 135
Gernat, T., Rao, V. D., Middendorf, M., Dankowicz, H. J., Goldenfeld, N. D., & Robinson, G. E. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1713568115 · Cited by 127
Collective Animal Behavior from Bayesian Estimation and Probability Matching
Pérez‐Escudero, A., & de Polavieja, G. G. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002282 · Cited by 121
Recognizing novel views of three-dimensional objects.
Humphrey, G. K., & Khan, S. C. (1992) · Canadian Journal of Psychology/Revue Canadienne de Psychologie · doi:10.1037/h0084320 · Cited by 116
Brembs, B., & Heisenberg, M. (2000) · Learning & Memory · doi:10.1101/lm.7.2.104 · Cited by 116
Pennekamp, F., Schtickzelle, N., & Petchey, O. L. (2015) · Ecology and Evolution · doi:10.1002/ece3.1529 · Cited by 114
FIMTrack: An open source tracking and locomotion analysis software for small animals
Risse, B., Berh, D., Otto, N., Klämbt, C., & Jiang, X. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005530 · Cited by 104
Social networks predict the life and death of honey bees
Wild, B., Dormagen, D. M., Zachariae, A., Smith, M. L., Traynor, K. S., Brockmann, D., Couzin, I. D., & Landgraf, T. (2021) · Nature Communications · doi:10.1038/s41467-021-21212-5 · Cited by 102
Self-citationCavagna, A., Cimarelli, A., Giardina, I., Parisi, G., Santagati, R., Stefanini, F., & Tavarone, R. (2010) · Mathematical Models and Methods in Applied Sciences · doi:10.1142/s0218202510004660 · Cited by 99
SLEAP: Multi-animal pose trackingPreprint
Pereira, T., Tabris, N., Li, J., Ravindranath, S., Papadoyannis, E. S., Wang, Z. Y., Turner, D. M., McKenzie‐Smith, G. C., Kocher, S. D., Falkner, A. L., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.31.276246 · Cited by 89
An automated barcode tracking system for behavioural studies in birds
Alarcón‐Nieto, G., Graving, J. M., Klarevas‐Irby, J. A., Maldonado‐Chaparro, A. A., Mueller, I., & Farine, D. R. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13005 · Cited by 84
Counting-Based Search: Branching Heuristics for Constraint Satisfaction Problems
Pesant, G., Quimper, C.-G., & Zanarini, A. (2012) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3463 · Cited by 65
Iwata, H., Ebana, K., Uga, Y., & Hayashi, T. (2015) · PLoS ONE · doi:10.1371/journal.pone.0120610 · Cited by 57
GroupTracker: Video tracking system for multiple animals under severe occlusion
Fukunaga, T., Kubota, S., Oda, S., & Iwasaki, W. (2015) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2015.02.006 · Cited by 40
Hewitt, B. M., Yap, M. H., Hodson-Tole, E. F., Kennerley, A. J., Sharp, P. S., & Grant, R. A. (2017) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2017.04.006 · Cited by 29
Closing the loop: tracking and perturbing behaviour of individuals in a group in real-timePreprint
Rasch, M. J., Shi, A., & Ji, Z. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/071308 · Cited by 7
Francisco, F. A., Nührenberg, P., & Jordan, A. L. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/571232 · Cited by 7
Cited by
SLEAP: A deep learning system for multi-animal pose tracking
Pereira, T. D., Tabris, N., Matsliah, A., Turner, D. M., Li, J., Ravindranath, S., Papadoyannis, E. S., Normand, E., Deutsch, D. S., Wang, Z. Y., et al. (2022) · Nature Methods · doi:10.1038/s41592-022-01426-1 · Cited by 886
Perspectives in machine learning for wildlife conservation
Tuia, D., Kellenberger, B., Beery, S., Costelloe, B. R., Zuffi, S., Risse, B., Mathis, A., Mathis, M. W., van Langevelde, F., Burghardt, T., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-27980-y · Cited by 740
Self-citationMulti-animal pose estimation, identification and tracking with DeepLabCut
Lauer, J., Zhou, M., Ye, S., Menegas, W., Schneider, S., Nath, T., Rahman, M. M., Di Santo, V., Soberanes, D., Feng, G., et al. (2022) · Nature Methods · doi:10.1038/s41592-022-01443-0 · Cited by 647
Towards the fully automated monitoring of ecological communities
Besson, M., Alison, J., Bjerge, K., Gorochowski, T. E., Høye, T. T., Jucker, T., Mann, H. M. R., & Clements, C. F. (2022) · Ecology Letters · doi:10.1111/ele.14123 · Cited by 275
Perspectives on Individual Animal Identification from Biology and Computer Vision
Vidal, M., Wolf, N., Rosenberg, B., Harris, B. P., & Mathis, A. (2021) · Integrative and Comparative Biology · doi:10.1093/icb/icab107 · Cited by 104
Metadata
- Authors
- Walter, T., & Couzin, I. D.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2021-02-26
- Volume
- 10
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W3131145322
- Cited by
- 286
- References
- 67
- FWCI
- 15.78
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.