Simple Behavioral Analysis (SimBA) – an open source toolkit for computer classification of complex social behaviors in experimental animals
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Aberrant social behavior is a core feature of many neuropsychiatric disorders, yet the study of complex social behavior in freely moving rodents is relatively infrequently incorporated into preclinical models. This likely contributes to limited translational impact. A major bottleneck for the adoption of socially complex, ethology-rich, preclinical procedures are the technical limitations for consistently annotating detailed behavioral repertoires of rodent social behavior. Manual annotation is subjective, prone to observer drift, and extremely time-intensive. Commercial approaches are expensive and inferior to manual annotation. Open-source alternatives often require significant investments in specialized hardware and significant computational and programming knowledge. By combining recent computational advances in convolutional neural networks and pose-estimation with further machine learning analysis, complex rodent social behavior is primed for inclusion under the umbrella of computational neuroethology. Here we present an open-source package with graphical interface and workflow (Simple Behavioral Analysis, SimBA) that uses pose-estimation to create supervised machine learning predictive classifiers of rodent social behavior, with millisecond resolution and accuracies that can out-perform human observers. SimBA does not require specialized video acquisition hardware nor extensive computational background. Standard descriptive statistical analysis, along with graphical region of interest annotation, are provided in addition to predictive classifier generation. To increase ease-of-use for behavioural neuroscientists, we designed SimBA with accessible menus for pre-processing videos, annotating behavioural training datasets, selecting advanced machine learning options, robust classifier validation functions and flexible visualizations tools. This allows for predictive classifier transparency, explainability and tunability prior to, and during, experimental use. We demonstrate that this approach is flexible and robust in both mice and rats by classifying social behaviors that are commonly central to the study of brain function and social motivation. Finally, we provide a library of poseestimation weights and behavioral predictive classifiers for resident-intruder behaviors in mice and rats. All code and data, together with detailed tutorials and documentation, are available on the SimBA GitHub repository . Graphical abstract: SimBA graphical interface (GUI) for creating supervised machine learning classifiers of rodent social behavior. (a) Pre-process videos . SimBA supports common video pre-processing functions (e.g., cropping, clipping, sampling, format conversion, etc.) that can be performed either on single videos, or as a batch. (b) Managing poseestimation data and creating classification projects . Pose-estimation tracking projects in DeepLabCut and DeepPoseKit can be either imported or created and managed within the SimBA graphical user interface, and the tracking results are imported into SimBA classification projects. SimBA also supports userdrawn region-of-interests (ROIs) for descriptive statistics of animal movements, or as features in machine learning classification projects. (c) Create classifiers, perform classifications, and analyze classification data . SimBA has graphical tools for correcting poseestimation tracking inaccuracies when multiple subjects are within a single frame, annotating behavioral events from videos, and optimizing machine learning hyperparameters and discrimination thresholds. A number of validation checkpoints and logs are included for increased classifier explainability and tunability prior to, and during, experimental use. Both detailed and summary data are provided at the end of classifier analysis. SimBA accepts behavioral annotations generated elsewhere (such as through JWatcher) that can be imported into SimBA classification projects. (d) Visualize classification results . SimBA has several options for visualizing machine learning classifications, animal movements and ROI data, and analyzing the durations and frequencies of classified behaviors. See the SimBA GitHub repository for a comprehensive documentation and user tutorials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
YOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · cited 2×
“We have conscientiously selected a supervised approach, and incorporated a number of explainability and validation checkpoints 38,39 , to ensure that researchers unfamiliar with these approaches are confident in the accuracy and validity of the classifiers they generate.”
BackgroundMethodHe, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×
“Additional advances in machine vision has also addressed identification issues through dual segmentation/identification networks 75,76 or by including measures of optical flow 77 and part affinity fields 73 .”
BackgroundLiu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · cited 1×
“Although the two correction tools eliminate unambiguous outliers, further performance improvements may be obtained by more refined and flexible machine learning methods 72–74 and/or data cleaning techniques that incorporates variable criteria for the different body-parts.”
BackgroundEvaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187 · cited 2×
“The classifiers generated for this manuscript use the DeepLabCut 27 default network architectures for pose-estimation positional data, which the subsequent machine learning features are derived from.”
MethodLearning from imbalanced data: open challenges and future directions
Krawczyk, B. (2016) · Progress in Artificial Intelligence · doi:10.1007/s13748-016-0094-0 · Cited by 2,459 · cited 1×
“SimBA includes a variety of statistical re-balancing tools that address class imbalances 65–68 .”
BackgroundA standardized protocol for repeated social defeat stress in mice
Golden, S. A., Covington, H. E., Berton, O., & Russo, S. J. (2011) · Nature Protocols · doi:10.1038/nprot.2011.361 · Cited by 1,581
Self-citationNatural neural projection dynamics underlying social behavior
Gunaydin, L. A., Grosenick, L., Finkelstein, J. C., Kauvar, I. V., Fenno, L. E., Adhikari, A., Lammel, S., Mirzabekov, J. J., Airan, R. D., Zalocusky, K. A., et al. (2014) · Cell · doi:10.1016/j.cell.2014.05.017 · Cited by 1,389
Explainable machine learning for scientific insights and discoveries
Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J. (2020) · IEEE Access · doi:10.1109/access.2020.2976199 · Cited by 1,040 · cited 2×
“We have conscientiously selected a supervised approach, and incorporated a number of explainability and validation checkpoints 38,39 , to ensure that researchers unfamiliar with these approaches are confident in the accuracy and validity of the classifiers they generate.”
BackgroundMethodFunctional identification of an aggression locus in the mouse hypothalamus
Lin, D., Boyle, M. P., Dollar, P., Lee, H., Lein, E. S., Perona, P., & Anderson, D. J. (2011) · Nature · doi:10.1038/nature09736 · Cited by 914 · cited 3×
“For these additional tracking models, we labelled a further 800 frames from home-cage environments with male rats, and 3200 CLAHE enhanced images from the Caltech Resident-Intruder Mouse (CRIM13) dataset 19,47 .”
BackgroundMethodSelf-citationMapping Sub-Second Structure in Mouse Behavior
Wiltschko, A. B., Johnson, M. J., Iurilli, G., Peterson, R. E., Katon, J. M., Pashkovski, S. L., Abraira, V. E., Adams, R. P., & Datta, S. R. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.031 · Cited by 760
Martin, L. A., Neighbors, H. W., & Griffith, D. M. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.1985 · Cited by 674 · cited 1×
“Maladaptive social behaviours are theorised as both a cause and consequence of many neuropsychiatric disorders including depression 1,2 , substance abuse 3 and PTSD 4–7 .”
BackgroundJAABA: Interactive machine learning for automatic annotation of animal behavior
Kabra, M., Robie, A. A., Rivera-Alba, M., Branson, S., & Branson, K. (2013) · Nature Methods · doi:10.1038/nmeth.2281 · Cited by 644
Mapping the stereotyped behaviour of freely moving fruit flies
Berman, G. J., Choi, D. M., Bialek, W., & Shaevitz, J. W. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0672 · Cited by 565
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Graving, J. M., Chae, D., Naik, H., Li, L., Koger, B., Costelloe, B. R., & Couzin, I. D. (2019) · eLife · doi:10.7554/elife.47994 · Cited by 547
Prefrontal cortical regulation of brainwide circuit dynamics and reward-related behavior
Ferenczi, E. A., Zalocusky, K. A., Liston, C., Grosenick, L., Warden, M. R., Amatya, D., Katovich, K., Mehta, H., Patenaude, B., Ramakrishnan, C., et al. (2016) · Science · doi:10.1126/science.aac9698 · Cited by 535
Computational Neuroethology: A Call to Action
Datta, S. R., Anderson, D. J., Branson, K., Perona, P., & Leifer, A. (2019) · Neuron · doi:10.1016/j.neuron.2019.09.038 · Cited by 453
Aggressive behavioral phenotypes in mice
Miczek, K. A., Maxson, S. C., Fish, E. W., & Faccidomo, S. (2001) · Behavioural Brain Research · doi:10.1016/s0166-4328(01)00298-4 · Cited by 448 · cited 1×
“Here we use SimBA to create a battery of 28 predictive classifiers for behavioral repertoires relevant for studying social motivations in rats and mice 62,63 .”
MethodAutomated monitoring and analysis of social behavior in Drosophila
Dankert, H., Wang, L., Hoopfer, E. D., Anderson, D. J., & Perona, P. (2009) · Nature Methods · doi:10.1038/nmeth.1310 · Cited by 382
Volitional social interaction prevents drug addiction in rat models
Venniro, M., Zhang, M., Caprioli, D., Hoots, J. K., Golden, S. A., Heins, C., Morales, M., Epstein, D. H., & Shaham, Y. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0246-6 · Cited by 342
Self-citationFacial expressions of emotion states and their neuronal correlates in mice
Dolensek, N., Gehrlach, D. A., Klein, A. S., & Gogolla, N. (2020) · Science · doi:10.1126/science.aaz9468 · Cited by 327
Time to connect: bringing social context into addiction neuroscience
Heilig, M., Epstein, D. H., Nader, M. A., & Shaham, Y. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.67 · Cited by 298 · cited 2×
“As such, the treatment of maladaptive social behaviour presents an important strategy for improving mental health 9–12 .”
BackgroundA size principle for recruitment of Drosophila leg motor neurons
Azevedo, A. W., Dickinson, E. S., Gurung, P., Venkatasubramanian, L., Mann, R. S., & Tuthill, J. C. (2020) · eLife · doi:10.7554/elife.56754 · Cited by 87
Cited by
SLEAP: A deep learning system for multi-animal pose tracking
Pereira, T. D., Tabris, N., Matsliah, A., Turner, D. M., Li, J., Ravindranath, S., Papadoyannis, E. S., Normand, E., Deutsch, D. S., Wang, Z. Y., et al. (2022) · Nature Methods · doi:10.1038/s41592-022-01426-1 · Cited by 886
Quantifying behavior to understand the brain
Pereira, T. D., Shaevitz, J. W., & Murthy, M. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-00734-z · Cited by 309
Identifying behavioral structure from deep variational embeddings of animal motion
Luxem, K., Mocellin, P., Fuhrmann, F., Kürsch, J., Miller, S. R., Palop, J. J., Remy, S., & Bauer, P. (2022) · Communications Biology · doi:10.1038/s42003-022-04080-7 · Cited by 218
“Supervised approaches like DeepEthogram, SimBa or MARS DeepEthogram, SimBa or MARS (Bohnslav et al., 2021; Nilsson et al., 2020; Segalin et al., 2021) all allow the labeling of episodes of interest.”
MethodOrganization of neural circuits underlying social behavior: A consideration of the medial amygdala
Raam, T., & Hong, W. (2021) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2021.02.008 · Cited by 134
Measuring behavior in the home cage: study design, applications, challenges, and perspectives
Grieco, F., Bernstein, B. J., Biemans, B., Bikovski, L., Burnett, C. J., Cushman, J. D., van Dam, E. A., Fry, S. A., Richmond-Hacham, B., Homberg, J. R., et al. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.735387 · Cited by 109
“Follow-up modules B-Soid (Hsu and Yttri, 2019) and SimBA (Nilsson et al., 2020) are now available to train behavior classifiers on body-part pose data.”
Background
Metadata
- Authors
- Nilsson, S. R., Goodwin, N. L., Choong, J. J., Hwang, S., Wright, H. R., Norville, Z. C., Tong, X., Lin, D., Bentzley, B. S., Eshel, N., McLaughlin, R. J., & Golden, S. A.
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2020-04-20
- OpenAlex
- W3016904080
- Cited by
- 212
- References
- 28
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.