Facemap: a framework for modeling neural activity based on orofacial tracking
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Recent studies in mice have shown that orofacial behaviors drive a large fraction of neural activity across the brain. To understand the nature and function of these signals, we need better computational models to characterize the behaviors and relate them to neural activity. Here we developed Facemap, a framework consisting of a keypoint tracker and a deep neural network encoder for predicting neural activity. Our algorithm for tracking mouse orofacial behaviors was more accurate than existing pose estimation tools, while the processing speed was several times faster, making it a powerful tool for real-time experimental interventions. The Facemap tracker was easy to adapt to data from new labs, requiring as few as 10 annotated frames for near-optimal performance. We used the keypoints as inputs to a deep neural network which predicts the activity of ~50,000 simultaneously-recorded neurons and, in visual cortex, we doubled the amount of explained variance compared to previous methods. Using this model, we found that the neuronal activity clusters that were well predicted from behavior were more spatially spread out across cortex. We also found that the deep behavioral features from the model had stereotypical, sequential dynamics that were not reversible in time. In summary, Facemap provides a stepping stone toward understanding the function of the brain-wide neural signals and their relation to behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0209-y · Cited by 5,187 · cited 12×
“We compared our model with current state-of-the-art tools for keypoint tracking, DeepLab-Cut and SLEAP [27, 31, 36].”
InfluentialSpontaneous behaviors drive multidimensional, brainwide activity
Stringer, C., Pachitariu, M., Steinmetz, N., Reddy, C. B., Carandini, M., & Harris, K. D. (2019) · Science · doi:10.1126/science.aav7893 · Cited by 1,510 · cited 9×
“Previous studies used simple approaches like reduced-rank regression or ridge regression [14, 20].”
InfluentialSelf-citationModulation of visual responses by behavioral state in mouse visual cortex
Niell, C. M., & Stryker, M. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.01.033 · Cited by 1,483
Arousal and Locomotion Make Distinct Contributions to Cortical Activity Patterns and Visual Encoding
Vinck, M., Batista-Brito, R., Knoblich, U., & Cardin, J. A. (2015) · Neuron · doi:10.1016/j.neuron.2015.03.028 · Cited by 819
Mapping Sub-Second Structure in Mouse Behavior
Wiltschko, A. B., Johnson, M. J., Iurilli, G., Peterson, R. E., Katon, J. M., Pashkovski, S. L., Abraira, V. E., Adams, R. P., & Datta, S. R. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.031 · Cited by 760
Cited by
Rapid fluctuations in functional connectivity of cortical networks encode spontaneous behavior
Benisty, H., Barson, D., Moberly, A. H., Lohani, S., Tang, L., Coifman, R. R., Crair, M. C., Mishne, G., Cardin, J. A., & Higley, M. J. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01498-y · Cited by 65
“Facial movements were derived from video data by using FaceMap 28 , and the first extracted component was used for most subsequent analyses.”
InfluentialVisual objects refine head direction coding
Siegenthaler, D., Denny, H., Carrasco, S. S., Mayer, J. L., Levenstein, D., Peyrache, A., Trenholm, S., & Macé, É. (2025) · Science · doi:10.1126/science.adu9828 · Cited by 16
Place and behavioral modulation of hippocampal neurons during immobility
Sartorato, N., Zouridis, I. S., Narantsatsralt, U.-U., Blanco-Hernández, E., & Burgalossi, A. (2026) · Nature Communications · doi:10.1038/s41467-026-75492-w · Cited by 1
“Pupil area and whisker-pad motion energy(ROIcenteredonthewhiskerpad)wereextractedandanalyzed with the open-source software Facemap (https://github.com/ MouseLand/facemap 19) in Python 121,122 .”
Metadata
- Authors
- Syeda, A., Zhong, L., Tung, R., Long, W., Pachitariu, M., & Stringer, C.
- Venue
- Nature Neuroscience
- Published
- 2023-11-20
- Volume
- 27
- Issue
- 1
- Pages
- 187-195
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W4388823868
- Cited by
- 129
- References
- 19
- FWCI
- 18.02
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Facemap: a framework for modeling neural activity based on orofacial tracking. https://app.cochat.ai/papers/10.1038/s41593-023-01490-6
- The paper
- Syeda, A., Zhong, L., Tung, R., Long, W., Pachitariu, M., & Stringer, C. (2024). Facemap: a framework for modeling neural activity based on orofacial tracking. Nature Neuroscience. https://doi.org/10.1038/s41593-023-01490-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.