JAABA: Interactive machine learning for automatic annotation of animal behavior
Open accessAbstract
We present a machine learning-based system for automatically computing interpretable, quantitative measures of animal behavior. Through our interactive system, users encode their intuition about behavior by annotating a small set of video frames. These manual labels are converted into classifiers that can automatically annotate behaviors in screen-scale data sets. Our general-purpose system can create a variety of accurate individual and social behavior classifiers for different organisms, including mice and adult and larval Drosophila.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Machine learning can tackle a wide range of tasks, including classifying observations into predefined sets, clustering data into groups that share an underlying process, and regressing an outcome against multiple factors and elucidating their contributory effect.
“ML can tackle a wide range of tasks, including classifying observations into predefined sets (Kabra, Robie, Rivera-Alba, Branson, & Branson, 2013), clustering data into groups that share an underlying process (Zhang, O'Reilly, Perry, Taylor, & Dennis, 2015) and regressing an outcome of interest against multiple factors and elucidating their contributory effect (Chesler, Wilson, Lariviere, Rodrig”
From Applications of machine learning in animal behaviour studies (2017) · cites this paper as Kabra et al., 2013
Machine learning is used to automate the classification of behaviours or activities and tracking movement trajectories from GPS, accelerometer and/or video data.
“ML is used to automate the classification of behaviours/activities (Kabra et al., 2013) and tracking movement trajectories (Dell et al., 2014).”
From Applications of machine learning in animal behaviour studies (2017) · cites this paper as Kabra et al., 2013
Machine learning can address different objectives in numerous distinct fields of animal behaviour and is becoming a staple approach for novel methods.
“It is clear that ML can address different objectives in numerous distinct fields of animal behaviour and is thus becoming a staple approach for novel methods (Dankert, Wang, Hoopfer, Anderson, & Perona, 2009;Kabra et al., 2013).”
From Applications of machine learning in animal behaviour studies (2017) · cites this paper as Kabra et al., 2013
A Janelia Automatic Animal Behavior Annotator classifier was used to quantify self-grooming.
“A Janelia Automatic Animal Behavior Annotator classifier42 was used to quantify self‐grooming (see Ref. 43).”
From Behavioral flexibility in a mouse model for obsessive‐compulsive disorder: Impaired Pavlovian reversal learning in SAPAP3 mutants (2019) · cites this paper as [42]
JAABA provides unbiased computer identification of behaviors.
“sent to JAABA (https://github.com/kristinbranson/JAABA; Kabra et al., 2013) Janelia Automatic Animal Behavior Annotator (RRID:SCR_027430) for unbiased computer identification of behaviors.”
From Functional imaging of nine distinct neuronal populations under a miniscope in freely behaving animals (2026) · cites this paper as Kabra et al., 2013
Cites
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884 · see citing works
Aggarwal, J. K., & Ryoo, M. S. (2011) · ACM Computing Surveys · doi:10.1145/1922649.1922653 · Cited by 2,053 · see citing works
A GAL4-Driver Line Resource for Drosophila Neurobiology
Jenett, A., Rubin, G. M., Ngo, T.-T. B., Shepherd, D., Murphy, C., Dionne, H., Pfeiffer, B. D., Cavallaro, A., Hall, D., Jeter, J., et al. (2012) · Cell Reports · doi:10.1016/j.celrep.2012.09.011 · Cited by 1,449 · see citing works
Tools for neuroanatomy and neurogenetics in Drosophila
Pfeiffer, B. D., Jenett, A., Hammonds, A. S., Ngo, T.-T. B., Misra, S., Murphy, C., Scully, A. L., Carlson, J. W., Wan, K. H., Laverty, T., et al. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0803697105 · Cited by 974 · see citing works
High-throughput ethomics in large groups of Drosophila
Branson, K. M., Robie, A. A., Bender, J. A., Perona, P., & Dickinson, M. H. (2009) · Nature Methods · doi:10.1038/nmeth.1328 · Cited by 781 · see citing works
Self-citationHigh-throughput behavioral analysis in C. elegans
Swierczek, N. A., Giles, A. C., Rankin, C. H., & Kerr, R. A. (2011) · Nature Methods · doi:10.1038/nmeth.1625 · Cited by 460 · see citing works
Automated monitoring and analysis of social behavior in Drosophila
Dankert, H., Wang, L., Hoopfer, E. D., Anderson, D. J., & Perona, P. (2009) · Nature Methods · doi:10.1038/nmeth.1310 · Cited by 382 · see citing works
Active sampling and decision making in Drosophila chemotaxis
Gómez-Marín, À., Stephens, G. J., & Louis, M. (2011) · Nature Communications · doi:10.1038/ncomms1455 · Cited by 255 · see citing works
Computerized video analysis of social interactions in mice
de Chaumont, F., Coura, R., Serreau, P., Cressant, A., Chabout, J., Granon, S., & Olivo‐Marín, J. (2012) · Nature Methods · doi:10.1038/nmeth.1924 · Cited by 238 · see citing works
A portrait of locomotor behaviour in Drosophila determined by a video-tracking paradigm
Martin, J. (2004) · Behavioural Processes · doi:10.1016/j.beproc.2004.04.003 · Cited by 160 · see citing works
Navigational Decision Making in Drosophila Thermotaxis
Luo, L., Gershow, M. H., Rosenzweig, M. R., Kang, K., Fang-Yen, C. M., Garrity, P., & Samuel, A. D. T. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.4090-09.2010 · Cited by 143 · see citing works
A New Chamber for Studying the Behavior of Drosophila
Simon, J. C., & Dickinson, M. H. (2010) · PLoS ONE · doi:10.1371/journal.pone.0008793 · Cited by 140 · see citing works
Robie, A. A., Straw, A. D., & Dickinson, M. H. (2010) · Journal of Experimental Biology · doi:10.1242/jeb.041749 · Cited by 70 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Mapping Sub-Second Structure in Mouse Behavior
Wiltschko, A. B., Johnson, M. J., Iurilli, G., Peterson, R. E., Katon, J. M., Pashkovski, S. L., Abraira, V. E., Adams, R. P., & Datta, S. R. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.031 · Cited by 760 · see citing works
A framework for studying emotions across species
Anderson, D. J., & Adolphs, R. (2014) · Cell · doi:10.1016/j.cell.2014.03.003 · Cited by 691 · see citing works
From bulk, single-cell to spatial RNA sequencing
Li, X., & Wang, C.-Y. (2021) · International Journal of Oral Science · doi:10.1038/s41368-021-00146-0 · Cited by 605 · see citing works
A multilevel multimodal circuit enhances action selection in drosophila
Ohyama, T., Schneider-Mizell, C. M., Fetter, R. D., Aleman, J. V., Franconville, R., Rivera-Alba, M., Mensh, B. D., Branson, K. M., Simpson, J. H., Truman, J. W., et al. (2015) · Nature · doi:10.1038/nature14297 · Cited by 502 · see citing works
Self-citationNeural circuit mechanisms of social behavior
Chen, P., & Hong, W. (2018) · Neuron · doi:10.1016/j.neuron.2018.02.026 · Cited by 527 · see citing works
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · see citing works
In toto imaging and reconstruction of post-implantation mouse development at the single-cell level
McDole, K., Guignard, L., Amat, F., Berger, A., Malandain, G., Royer, L. A., Turaga, S. C., Branson, K., & Keller, P. J. (2018) · Cell · doi:10.1016/j.cell.2018.09.031 · Cited by 522 · see citing works
Self-citationApplications of machine learning in animal behaviour studies
Valletta, J. J., Torney, C., Kings, M., Thornton, A., & Madden, J. (2017) · Animal Behaviour · doi:10.1016/j.anbehav.2016.12.005 · Cited by 505 · see citing works
“ML is used to automate the classification of behaviours/activities (Kabra et al., 2013) and tracking movement trajectories (Dell et al., 2014).”
Automated image-based tracking and its application in ecology
Dell, A. I., Bender, J. A., Branson, K., Couzin, I. D., de Polavieja, G. G., Noldus, L. P. J. J., Pérez-Escudero, A., Perona, P., Straw, A. D., Wikelski, M., et al. (2014) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2014.05.004 · Cited by 492 · see citing works
Computational Neuroethology: A Call to Action
Datta, S. R., Anderson, D. J., Branson, K., Perona, P., & Leifer, A. (2019) · Neuron · doi:10.1016/j.neuron.2019.09.038 · Cited by 453 · see citing works
Self-citationCombined Social and Spatial Coding in a Descending Projection from the Prefrontal Cortex
Murugan, M., Jang, H. J., Park, M., Miller, E. M., Cox, J., Taliaferro, J. P., Parker, N. F., Bhave, V., Hur, H., Liang, Y., et al. (2017) · Cell · doi:10.1016/j.cell.2017.11.002 · Cited by 311 · see citing works
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343 · see citing works
Structure of the Zebrafish Locomotor Repertoire Revealed with Unsupervised Behavioral Clustering
Marques, J. C., Lackner, S., Félix, R., & Orger, M. B. (2018) · Current Biology · doi:10.1016/j.cub.2017.12.002 · Cited by 245 · see citing works
Weissbrod, A., Shapiro, A., Vasserman, G., Edry, L., Dayan, M., Yitzhaky, A., Hertzberg, L., Feinerman, O., & Kimchi, T. (2013) · Nature Communications · doi:10.1038/ncomms3018 · Cited by 235 · see citing works
A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives
Mathis, A., Schneider, S., Lauer, J., & Mathis, M. W. (2020) · Neuron · doi:10.1016/j.neuron.2020.09.017 · Cited by 231 · see citing works
Female mice ultrasonically interact with males during courtship displays
Neunuebel, J. P., Taylor, A. L., Arthur, B. J., & Egnor, S. R. (2015) · eLife · doi:10.7554/elife.06203 · Cited by 224 · see citing works
Ventral hippocampal projections to the medial prefrontal cortex regulate social memory
Phillips, M. L., Robinson, H. A., & Pozzo-Miller, L. (2019) · eLife · doi:10.7554/elife.44182 · Cited by 205 · see citing works
Cell types and neuronal circuitry underlying female aggression in Drosophila
Schretter, C. E., Aso, Y., Robie, A. A., Dreher, M., Dolan, M.-J., Chen, N., Ito, M., Yang, T., Parekh, R., Branson, K. M., et al. (2020) · eLife · doi:10.7554/elife.58942 · Cited by 125 · see citing works
Self-citationThe neural basis for a persistent internal state in Drosophila females
Deutsch, D., Pacheco, D., Encarnacion-Rivera, L., Pereira, T., Fathy, R., Clemens, J., Girardin, C., Calhoun, A., Ireland, E., Burke, A., et al. (2020) · eLife · doi:10.7554/elife.59502 · Cited by 115 · see citing works
Generating parallel representations of position and identity in the olfactory system
Taisz, I., Donà, E., Münch, D., Bailey, S. N., Morris, B. J., Meechan, K. I., Stevens, K. M., Varela-Martínez, I., Gkantia, M., Schlegel, P., et al. (2023) · Cell · doi:10.1016/j.cell.2023.04.038 · Cited by 79 · see citing works
Threshold-based ordering of sequential actions during Drosophila courtship
McKellar, C. E., Lillvis, J. L., Bath, D. E., Fitzgerald, J. E., Cannon, J. G., Simpson, J. H., & Dickson, B. J. (2019) · Current Biology · doi:10.1016/j.cub.2018.12.019 · Cited by 67 · see citing works
Automated classification of self-grooming in mice using open-source software
van den Boom, B. J. G., Pavlidi, P., Wolf, C. J. H., Mooij, A. H., & Willuhn, I. (2017) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2017.05.026 · Cited by 62 · see citing works
Fujiwara, T., Brotas, M., & Chiappe, M. E. (2022) · Neuron · doi:10.1016/j.neuron.2022.04.008 · Cited by 59 · see citing works
Mezzera, C., Brotas, M., Gaspar, M., Pavlou, H. J., Goodwin, S. F., & Vasconcelos, M. L. (2020) · Current Biology · doi:10.1016/j.cub.2020.06.071 · Cited by 55 · see citing works
van den Boom, B. J. G., Mooij, A. H., Misevičiūtė, I., Denys, D., & Willuhn, I. (2019) · Genes, Brain and Behavior · doi:10.1111/gbb.12557 · Cited by 55 · see citing works
Dynamic structure of locomotor behavior in walking fruit flies
Katsov, A. Y., Freifeld, L., Horowitz, M., Kuehn, S., & Clandinin, T. R. (2017) · eLife · doi:10.7554/elife.26410 · Cited by 43 · see citing works
A locomotor neural circuit persists and functions similarly in larvae and adult Drosophila
Lee, K., & Doe, C. Q. (2021) · eLife · doi:10.7554/elife.69767 · Cited by 49 · see citing works
Tools for rapid high-resolution behavioral phenotyping of automatically isolated Drosophila
Williamson, W. R., Peek, M. Y., Breads, P., Coop, B., & Card, G. M. (2018) · Cell Reports · doi:10.1016/j.celrep.2018.10.048 · Cited by 36 · see citing works
Social state alters vision using three circuit mechanisms in Drosophila
Schretter, C. E., Hindmarsh Sten, T., Klapoetke, N., Shao, M., Nern, A., Dreher, M., Bushey, D., Robie, A. A., Taylor, A. L., Branson, K., et al. (2025) · Nature · doi:10.1038/s41586-024-08255-6 · Cited by 29 · see citing works
Self-citationA subcortical switchboard for perseverative, exploratory and disengaged states
Ahmadlou, M., Shirazi, M. Y., Zhang, P., Rogers, I. L. M., Dziubek, J., Young, M., & Hofer, S. B. (2025) · Nature · doi:10.1038/s41586-025-08672-1 · Cited by 25 · see citing works
Spinal Inhibitory Ptf1a-Derived Neurons Prevent Self-Generated Itch
Escalante, A., & Klein, R. (2020) · Cell Reports · doi:10.1016/j.celrep.2020.108422 · Cited by 17 · see citing works
Phillips, M. L., Urban, N. T., Salemi, T., Dong, Z., & Yasuda, R. (2026) · eLife · doi:10.7554/elife.110277.3 · Cited by 1 · see citing works
- Authors
- Kabra, M., Robie, A. A., Rivera-Alba, M., Branson, S., & Branson, K.
- Affiliations
- Howard Hughes Medical Institute (US)
- Corresponding author
- Branson, Kristin
- Venue
- Nature Methods
- Published
- 2013-01
- Volume
- 10
- Issue
- 1
- Pages
- 64-67
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- OpenAlex
- W1977591411
- Cited by
- 644
- References
- 13
- Citations by year
- 2026: 51 · 2025: 52 · 2024: 65 · 2023: 86 · 2022: 63 · 2021: 59 · 2020: 59 · 2019: 53 · 2018: 39 · 2017: 33 · 2016: 23 · 2015: 25 · 2014: 24 · 2013: 11
- FWCI
- 19.75
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- www.springer.com/tdm (tdm)
- Locations
- Nature Methods (publishedVersion)
- PubMed (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- Howard Hughes Medical Institute
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurobiology and Insect Physiology Research
- Keywords
- behavior classification, interactive machine learning, video annotation, behavioral tracking, mouse behavior, Drosophila behavior, larval Drosophila, social behavior, open source software
- MeSH terms
- Algorithms, Animals, Artificial Intelligence, Behavior, Animal, Diagnosis, Computer-Assisted, Drosophila melanogaster, Larva, Mice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
