Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
The study of complex behaviors is often challenging when using manual annotation due to the absence of quantifiable behavioral definitions and the subjective nature of behavioral annotation. Integration of supervised machine learning approaches mitigates some of these issues through the inclusion of accessible and explainable model interpretation. To decrease barriers to access, and with an emphasis on accessible model explainability, we developed the open-source Simple Behavioral Analysis (SimBA) platform for behavioral neuroscientists. SimBA introduces several machine learning interpretability tools, including SHapley Additive exPlanation (SHAP) scores, that aid in creating explainable and transparent behavioral classifiers. Here we show how the addition of explainability metrics allows for quantifiable comparisons of aggressive social behavior across research groups and species, reconceptualizing behavior as a sharable reagent and providing an open-source framework. We provide an open-source, graphical user interface (GUI)-driven, well-documented package to facilitate the movement toward improved automation and sharing of behavioral classification tools across laboratories. This paper presents Simple Behavioral Analysis (SimBA), an open-source platform for automated, explainable machine learning analysis of behavior. SimBA comes with extensive documentation, a graphical interface and an active community and works with any organism tracked by pose estimation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · cited 1×
“Explanations for how machine learning models reach their decisions can help researchers communicate and compare the results of disparate classifiers and support researchers in making informed decisions for machine model implementation and use 51–54 .”
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187 · cited 4×
“We created pose estimation models in DeepLabCut (version 1.0) 10 for rats, the CRIM dataset and our experimental mouse videos.”
InfluentialAxiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
A standardized protocol for repeated social defeat stress in mice
Golden, S. A., Covington, H. E., Berton, O., & Russo, S. J. (2011) · Nature Protocols · doi:10.1038/nprot.2011.361 · Cited by 1,581 · cited 2×
“Alternately, CSDS testing is typically performed in thinner subdivided hamster cages, where the ‘resident’ animal is an established aggressor 62 and the intruder is a smaller mouse that experiences up to 10 consecutive days of defeat.”
Self-citationConsistent Individualized Feature Attribution for Tree Ensembles
Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2018) · arXiv · doi:10.48550/arxiv.1802.03888 · Cited by 1,430
Natural neural projection dynamics underlying social behavior
Gunaydin, L. A., Grosenick, L., Finkelstein, J. C., Kauvar, I. V., Fenno, L. E., Adhikari, A., Lammel, S., Mirzabekov, J. J., Airan, R. D., Zalocusky, K. A., et al. (2014) · Cell · doi:10.1016/j.cell.2014.05.017 · Cited by 1,389 · cited 1×
“These caveats inhibit the detailed study of complex social repertoires in larger datasets and notably provide lower temporal resolution than most modern methodologies, such as in vivo electrophysiological, fiber photometry and single-cell calcium endomicroscopy recordings 5–8 .”
Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387 · cited 1×
“A further challenge for the field is appreciating how to accurately judge the performance of machine learning models, which can be biased toward subsets of animals and recording environment and that depend on timeseries data that are vulnerable to leakage 77 .”
Neuroscience Needs Behavior: Correcting a Reductionist Bias
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A., & Poeppel, D. (2017) · Neuron · doi:10.1016/j.neuron.2016.12.041 · Cited by 1,358 · cited 1×
“Behavioral neuroscience requires detailed behavior 1 , but the notoriously painstaking process of hand-annotating live or recorded assays poses a substantial bottleneck preventing comprehensive behavioral analysis.”
JAABA: Interactive machine learning for automatic annotation of animal behavior
Kabra, M., Robie, A. A., Rivera-Alba, M., Branson, S., & Branson, K. (2013) · Nature Methods · doi:10.1038/nmeth.2281 · Cited by 644
Mapping the stereotyped behaviour of freely moving fruit flies
Berman, G. J., Choi, D. M., Bialek, W., & Shaevitz, J. W. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0672 · Cited by 565
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning
Graving, J. M., Chae, D., Naik, H., Li, L., Koger, B., Costelloe, B. R., & Couzin, I. D. (2019) · eLife · doi:10.7554/elife.47994 · Cited by 547
Prefrontal cortical regulation of brainwide circuit dynamics and reward-related behavior
Ferenczi, E. A., Zalocusky, K. A., Liston, C., Grosenick, L., Warden, M. R., Amatya, D., Katovich, K., Mehta, H., Patenaude, B., Ramakrishnan, C., et al. (2016) · Science · doi:10.1126/science.aac9698 · Cited by 535
Computational Neuroethology: A Call to Action
Datta, S. R., Anderson, D. J., Branson, K., Perona, P., & Leifer, A. (2019) · Neuron · doi:10.1016/j.neuron.2019.09.038 · Cited by 453
Automated monitoring and analysis of social behavior in Drosophila
Dankert, H., Wang, L., Hoopfer, E. D., Anderson, D. J., & Perona, P. (2009) · Nature Methods · doi:10.1038/nmeth.1310 · Cited by 382
Epigenetic regulation of RAC1 induces synaptic remodeling in stress disorders and depression
Golden, S. A., Christoffel, D. J., Heshmati, M., Hodes, G. E., Magida, J., Davis, K., Cahill, M. E., Dias, C., Ribeiro, E., Ables, J. L., et al. (2013) · Nature Medicine · doi:10.1038/nm.3090 · Cited by 322 · cited 1×
“No statistical methods were used to pre-determine sample sizes, but our sample sizes are similar to those reported in previous publications 60,62,97 .”
Self-citationCounterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review
Verma, S., Boonsanong, V., Hoang, M., Hines, K. E., Dickerson, J. P., & Shah, C. (2020) · arXiv · doi:10.48550/arxiv.2010.10596 · Cited by 267 · cited 1×
“Shapley values are just one paradigm among a continuously growing number of approaches for explainable artificial intelligence (XAI); alternate options include LIME 52 and counterfactuals 56 , all with different pros and cons.”
Establishment of a repeated social defeat stress model in female mice
Takahashi, A., Chung, J.-R., Zhang, S., Zhang, H., Grossman, Y., Aleyasin, H., Flanigan, M. E., Pfau, M. L., Menard, C., Dumitriu, D., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-12811-8 · Cited by 256
Identifying behavioral structure from deep variational embeddings of animal motion
Luxem, K., Mocellin, P., Fuhrmann, F., Kürsch, J., Miller, S. R., Palop, J. J., Remy, S., & Bauer, P. (2022) · Communications Biology · doi:10.1038/s42003-022-04080-7 · Cited by 218
Newton, K. C., Kacev, D., Nilsson, S. R. O., Saettele, A. L., Golden, S. A., & Sheets, L. (2023) · Communications Biology · doi:10.1038/s42003-023-04449-2 · Cited by 36 · cited 2×
“…battery of runtime-optimized feature calculators covering frequentist and circular statistics, anomaly scores, temporal and spectral analyses, relationships between pose estimation and user-defined ROIs, bounding box methods and other machine learning distribution comparison techniques 18,30,39 .”
Self-citationRigney, N., Zbib, A., de Vries, G. J., & Petrulis, A. (2022) · Journal of Neuroendocrinology · doi:10.1111/jne.13083 · Cited by 29
Cited by
Evaluating the play fighting of rats: A sociological perspective
Ham, J. R., Pellis, S. M., Pellis, V. C., & Achterberg, E. J. M. (2026) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2026.110836 · Cited by 2
Metadata
- Authors
- Goodwin, N. L., Choong, J. J., Hwang, S., Pitts, K., Bloom, L., Islam, A., Zhang, Y. Y., Szelenyi, E. R., Tong, X., Newman, E. L., Miczek, K., Wright, H. R., McLaughlin, R. J., Norville, Z. C., Eshel, N., Heshmati, M., Nilsson, S. R. O., & Golden, S. A.
- Venue
- Nature Neuroscience
- Published
- 2024-05-22
- Volume
- 27
- Issue
- 7
- Pages
- 1411-1424
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W4398209670
- Cited by
- 198
- References
- 23
- FWCI
- 29.17
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.