Towards image-based animal tracking in natural environments using a freely moving camera
Abstract
Background Image-based tracking of individual animals can provide rich data to underpin breakthroughs in biological and medical research, but few if any existing methods extend to tracking unconstrained natural behaviour in the field.New method We have developed a visual tracking system for animals filmed with a freely moving hand-held or drone-operated camera in their natural environment. This exploits a global inference method for detecting motion of an animal against a cluttered background. Trajectories are then generated by a novel video key-frame selection scheme in combination with a geometrically constrained image stitching algorithm, resulting in a two-dimensional panorama image of the environment on which the dense animal path is displayed.Results By introducing a minimal and plausible set of constraints regarding the camera orientation and movement, we demonstrate that both per-frame animal positions and overall trajectories can be extracted with reasonable accuracy, for a range of different animals, environments and imaging modalities.Comparison Our method requires only a single uncalibrated camera, does not require marking or training data to detect the animal, and makes no prior assumptions about appearance of the target or background. In particular it can detect targets occupying fewer than 20 pixels in the image, and deal with poor contrast, highly dynamic lighting and frequent occlusion.Conclusion Our algorithm produces highly informative qualitative trajectories embedded in a panorama of the environment. The results are still subject to rotational drift and additional scaling routines would be needed to obtain absolute real-world coordinates. It nevertheless provides a flexible and easy-to-use system to obtain rich data on natural animal behaviour in the field.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies have shown that animal observations like trajectories highly benefit from additional cues included in the environmental context.
“Studies have shown that animal observations like trajectories highly benefit from additional cues included in the environmental context112.”
From Perspectives in machine learning for wildlife conservation (2022) · cites this paper as [112]
Cites
ORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037 · see citing works
Defaunation in the Anthropocene
Dirzo, R., Young, H. S., Galetti, M., Ceballos, G., Isaac, N. J. B., & Collen, B. (2014) · Science · doi:10.1126/science.1251817 · Cited by 4,305 · cited 1× · see citing works
“Most behavioural quantifications for these studies are still done manually; the need for novel automatic method-15 ologies is emphasised in multiple publications [1, 7, 8, 9].”
BackgroundAutomatic Panoramic Image Stitching using Invariant Features
Brown, M., & Lowe, D. G. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-0002-3 · Cited by 2,659 · see citing works
Terrestrial animal tracking as an eye on life and planet
Kays, R., Crofoot, M. C., Jetz, W., & Wikelski, M. (2015) · Science · doi:10.1126/science.aaa2478 · Cited by 1,638 · see citing works
Visual Tracking: An Experimental Survey
Smeulders, A. W. M., Chu, D. M., Cucchiara, R., Calderara, S., Dehghan, A., & Shah, M. A. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.230 · Cited by 1,527 · see citing works
Learning to Track at 100 FPS with Deep Regression Networks
Held, D., Thrun, S., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_45 · Cited by 1,199 · see citing works
Light Pollution as a Biodiversity Threat
Hölker, F., Wolter, C., Perkin, E. K., & Tockner, K. (2010) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2010.09.007 · Cited by 860 · see citing works
The Visual Object Tracking VOT2016 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin, L., Vojír, T., Häger, G., Lukežič, A., Fernandez, G., et al. (2016) · ECCV Workshops · doi:10.1007/978-3-319-48881-3_54 · Cited by 776 · see citing works
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Graving, J. M., Chae, D., Naik, H., Li, L., Koger, B., Costelloe, B. R., & Couzin, I. D. (2019) · eLife · doi:10.7554/elife.47994 · Cited by 547 · see citing works
Automated image-based tracking and its application in ecology
Dell, A. I., Bender, J. A., Branson, K., Couzin, I. D., de Polavieja, G. G., Noldus, L. P. J. J., Pérez-Escudero, A., Perona, P., Straw, A. D., Wikelski, M., et al. (2014) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2014.05.004 · Cited by 492 · see citing works
idtracker.ai: tracking all individuals in small or large collectives of unmarked animals
Romero-Ferrero, F., Bergomi, M. G., Hinz, R. C., Heras, F. J. H., & de Polavieja, G. G. (2019) · Nature Methods · doi:10.1038/s41592-018-0295-5 · Cited by 417 · see citing works
Scale Drift-Aware Large Scale Monocular SLAM
Strasdat, H., Montiel, J. M. M., & Davison, A. (2010) · Robotics: Science and Systems · doi:10.15607/rss.2010.vi.010 · Cited by 396 · see citing works
Neonicotinoids Interfere with Specific Components of Navigation in Honeybees
Fischer, J., Müller, T., Spatz, A.-K., Greggers, U., Grünewald, B., & Menzel, R. (2014) · PLoS ONE · doi:10.1371/journal.pone.0091364 · Cited by 340 · see citing works
Kellenberger, B., Marcos, D., & Tuia, D. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.06.028 · Cited by 335 · see citing works
Scale Drift-Aware Large Scale Monocular SLAM
Strasdat, H., Montiel, J. M. M., & Davison, A. J. (2011) · The MIT Press eBooks · doi:10.7551/mitpress/9123.003.0014 · Cited by 320 · see citing works
The Visual Object Tracking VOT2017 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Zajc, L. Č., Vojír, T., Häger, G., Lukežič, A., Eldesokey, A., et al. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.230 · Cited by 319 · see citing works
Computational Analysis of Behavior
Egnor, S. E. R., & Branson, K. M. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-013845 · Cited by 189 · see citing works
Hammer, T. J., Fierer, N., Hardwick, B., Simojoki, A., Slade, E. M., Taponen, J., Viljanen, H., & Roslin, T. V. (2016) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2016.0150 · Cited by 91 · see citing works
Performance of an insect-inspired target tracker in natural conditions
Bagheri, Z. M., Wiederman, S. D., Cazzolato, B. S., Grainger, S., & O’Carroll, D. C. (2017) · Bioinspiration & Biomimetics · doi:10.1088/1748-3190/aa5b48 · Cited by 53 · see citing works
Visual Tracking of Small Animals in Cluttered Natural Environments Using a Freely Moving Camera
Risse, B., Mangan, M., Pero, L. D., & Webb, B. H. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.335 · Cited by 42 · see citing works
Self-citationAnimal telemetry: Tagging effects
McIntyre, T. (2015) · Science · doi:10.1126/science.349.6248.596-b · Cited by 20 · see citing works
A Convolutional Neural Network for Automatic Analysis of Aerial Imagery
Maire, F., Mejías, L., & Hodgson, A. J. (2014) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2014.7008084 · Cited by 18 · see citing works
idtracker.ai: Tracking all individuals in large collectives of unmarked animalsPreprint
Romero-Ferrero, F., Bergomi, M. G., Hinz, R. C., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/280735 · Cited by 16 · see citing works
Animal telemetry: Follow the insects
Kissling, W. D. (2015) · Science · doi:10.1126/science.349.6248.597 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Perspectives in machine learning for wildlife conservation
Tuia, D., Kellenberger, B., Beery, S., Costelloe, B. R., Zuffi, S., Risse, B., Mathis, A., Mathis, M. W., van Langevelde, F., Burghardt, T., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-27980-y · Cited by 740 · see citing works
Self-citationKoger, B., Deshpande, A., Kerby, J. T., Graving, J. M., Costelloe, B. R., & Couzin, I. D. (2023) · Journal of Animal Ecology · doi:10.1111/1365-2656.13904 · Cited by 135 · see citing works
Evolution of central neural circuits: state of the art and perspectives
Roberts, R. J. V., Pop, S., & Prieto-Godino, L. L. (2022) · Nature reviews. Neuroscience · doi:10.1038/s41583-022-00644-y · Cited by 108 · see citing works
Ratnayake, M. N., Dyer, A. G., & Dorin, A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0239504 · Cited by 84 · see citing works
Animal behavior analysis methods using deep learning: A survey
Fazzari, E., Romano, D., Falchi, F., & Stefanini, C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128330 · Cited by 71 · see citing works
Methods for Monitoring Large Terrestrial Animals in the Wild
Prosekov, A. Y., Kuznetsov, A., Rada, A. O., & Ivanova, S. A. (2020) · Forests · doi:10.3390/f11080808 · Cited by 65 · see citing works
Spatial Monitoring and Insect Behavioural Analysis Using Computer Vision for Precision Pollination
Ratnayake, M. N., Amarathunga, D. C., Zaman, A., Dyer, A. G., & Dorin, A. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01715-4 · Cited by 53 · see citing works
CATER: Combined Animal Tracking & Environment Reconstruction
Haalck, L., Mangan, M., Wystrach, A., Clément, L., Webb, B. H., & Risse, B. (2023) · Science Advances · doi:10.1126/sciadv.adg2094 · Cited by 42 · see citing works
Self-citationAn examination of trends in the growing scientific literature on approaching wildlife with drones
Mo, M., & Bonatakis, K. (2022) · Drone Systems and Applications · doi:10.1139/dsa-2021-0003 · Cited by 35 · see citing works
Towards Computer Vision and Deep Learning Facilitated Pollination Monitoring for Agriculture
Ratnayake, M. N., Dyer, A. G., & Dorin, A. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00327 · Cited by 30 · see citing works
Through Hawks’ Eyes: Synthetically Reconstructing the Visual Field of a Bird in Flight
Miñano, S., Golodetz, S., Cavallari, T., & Taylor, G. K. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-022-01733-2 · Cited by 23 · see citing works
WATB: Wild Animal Tracking Benchmark
Wang, F., Ping, C., Li, F., Wang, X., He, B., & Sun, F. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01732-3 · Cited by 22 · see citing works
Advanced intelligent monitoring technologies for animals: A survey
Xu, P., Zhang, Y., Ji, M., Guo, S., Tang, Z., Wang, X., Guo, J., Zhang, J., & Guan, Z. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127640 · Cited by 21 · see citing works
Stöckl, A., & Foster, J. J. (2022) · Frontiers in Cellular Neuroscience · doi:10.3389/fncel.2022.984282 · Cited by 19 · see citing works
Smith, T. J., Smith, T. R., Faruk, F., Bendea, M., Kumara, S. T., Capadona, J. R., Hernandez-Reynoso, A. G., & Pancrazio, J. J. (2024) · eNeuro · doi:10.1523/eneuro.0500-23.2024 · Cited by 14 · see citing works
Leveraging big data to uncover the eco-evolutionary factors shaping behavioural development
Ehlman, S. M., Scherer, U., Bierbach, D., Francisco, F. A., Laskowski, K. L., Krause, J., & Wolf, M. (2023) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2022.2115 · Cited by 16 · see citing works
Towards a Dynamic Vision Sensor-based Insect Camera Trap
Gebauer, E., Thiele, S., Ouvrard, P., Sicard, A., & Risse, B. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00700 · Cited by 12 · see citing works
Self-citationLiu, R., Zhu, J., & Rao, X. (2022) · Symmetry · doi:10.3390/sym14071340 · Cited by 11 · see citing works
Compensation to visual impairments and behavioral plasticity in navigating ants
Schwarz, S., Clément, L., Haalck, L., Risse, B., & Wystrach, A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2410908121 · Cited by 11 · see citing works
Self-citationSun, G., Shi, C., Liu, J., Ma, P., & Ma, J. (2021) · IEEE Access · doi:10.1109/access.2021.3075272 · Cited by 11 · see citing works
Straw, A. D. (2021) · Integrative and Comparative Biology · doi:10.1093/icb/icab126 · Cited by 11 · see citing works
Over 300 km Dispersion of Wild Boar during Hot Summer, from Central Poland to Ukraine
Popczyk, B., Klich, D., Nasiadka, P., Nieszała, A., Gadkowski, K., Sobczuk, M., Balcerak, M., Kociuba, P., Olech, W., & Purski, L. (2024) · Animals · doi:10.3390/ani14010170 · Cited by 10 · see citing works
Gallagher, J., Gogia, A., & Oughton, E. J. (2024) · IEEE Access · doi:10.1109/access.2024.3522236 · Cited by 11 · see citing works
WildDrone: autonomous drone technology for monitoring wildlife populations
Schultz, U. P., Afridi, S., Berthelot, C., Ngoc, D. N., Hlebowicz, K., Iannino, E., Laporte‐Devylder, L., Maalouf, G., May, G. B., Meier, K., et al. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1695319 · Cited by 8 · see citing works
Self-citationSaha, R., Jiang, L., Salsabili, H., Faezipour, M., Ostadabbas, S., Larimer, B. M., & Mirbozorgi, S. A. (2023) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2023.3284823 · Cited by 7 · see citing works
Analysis of Racing Greyhound Path Following Dynamics Using a Tracking System
Eager, D., Hossain, I., Ishac, K., & Robins, S. (2021) · Animals · doi:10.3390/ani11092687 · Cited by 5 · see citing works
Koger, B., Deshpande, A., Kerby, J. T., Graving, J. M., Costelloe, B. R., & Couzin, I. D. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.06.30.498251 · Cited by 5 · see citing works
Haalck, L., & Risse, B. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00189 · Cited by 3 · see citing works
Self-citationEzenwa, I. M., Orakwelu, C. H., Okechukwu, C. R., Ebido, C. C., & Martin, R. O. (2024) · Journal für Ornithologie · doi:10.1007/s10336-024-02219-3 · Cited by 3 · see citing works
Borgne, H. L., & Bouget, C. (2023) · Naturae · doi:10.5852/naturae2023a6 · Cited by 3 · see citing works
Interactive machine vision for wildlife conservation
Kellenberger, B. (2020) · doi:10.18174/511122 · Cited by 3 · see citing works
Applications of Accelerometers and Other Bio-Logging Devices in Captive and Wild Animals
Campera, M., Chimienti, M., & Nekaris, K. A. I. (2023) · Animals · doi:10.3390/ani13020222 · Cited by 3 · see citing works
Automated Animal Tracking for Behavioral Experiments
Kong, M., & Peng, R. (2022) · International Conference on Biomedical and Bioinformatics Engineering (ICBBE) · doi:10.1145/3574198.3574243 · Cited by 2 · see citing works
de Margerie, E., & Monmasson, K. (2025) · Peer Community Journal · doi:10.24072/pcjournal.566 · Cited by 2 · see citing works
Gallagher, J., Gogia, A., & Oughton, E. J. (2024) · Research Square · doi:10.21203/rs.3.rs-4518266/v1 · Cited by 2 · see citing works
A Review on Object Tracking Across Real-World Multi Camera Environment
Cherian, R., Jothimani, K., & Reeja, S. R. (2021) · International Journal of Computer Applications · doi:10.5120/ijca2021921007 · Cited by 1 · see citing works
Sentinel animals : Enriching artificial intelligence with wildlife ecology to guard rhinos
Eikelboom, J. A. J. (2021) · doi:10.18174/544693 · Cited by 1 · see citing works
Observation and Classification of Fauna for Forest Survey
Naveen, V. (2022) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2022.45853
Solving the Plane-Sphere Ambiguity in Top-Down Structure-from-Motion
Haalck, L., & Risse, B. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00345
Self-citationAbbar, F. A., Eikelboom, J. A. J., Fiori, L., Visser, F., Grundlehner, A., Lodder, A., Costelloe, B. R., Hollands, R., Aarts, G. M., & van Langevelde, F. (2026) · Marine Mammal Science · doi:10.1111/mms.70268
- Authors
- Haalck, L., Mangan, M., Webb, B., & Risse, B.
- Affiliations
- University of Münster (DE)
- University of Sheffield (GB)
- University of Edinburgh (GB)
- Corresponding author
- Risse, Benjamin
- Venue
- Journal of Neuroscience Methods
- Publisher
- Elsevier BV
- Published
- 2020-01
- Volume
- 330
- Pages
- 108455
- ISSN
- 0165-0270 (ISSN-L)
- OpenAlex
- W2983108474
- Cited by
- 41
- References
- 24
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 9 · 2023: 7 · 2022: 10 · 2021: 8 · 2020: 3
- FWCI
- 2.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Neuroscience Methods (publishedVersion)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Funders
- Microsoft
- Engineering and Physical Sciences Research Council
- Heinrich Böll Stiftung
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- animal tracking, image tracking, drone camera, image stitching, animal trajectories, motion detection, background clutter, occlusion, uncalibrated cameras, natural behavior, field observations
- MeSH terms
- Algorithms, Animals, Behavior, Animal, Environment, Movement, Neurosciences, Video Recording
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
