The Code for Facial Identity in the Primate Brain
Open accessAbstract
Primates recognize complex objects such as faces with remarkable speed and reliability. Here, we reveal the brain's code for facial identity. Experiments in macaques demonstrate an extraordinarily simple transformation between faces and responses of cells in face patches. By formatting faces as points in a high-dimensional linear space, we discovered that each face cell's firing rate is proportional to the projection of an incoming face stimulus onto a single axis in this space, allowing a face cell ensemble to encode the location of any face in the space. Using this code, we could precisely decode faces from neural population responses and predict neural firing rates to faces. Furthermore, this code disavows the long-standing assumption that face cells encode specific facial identities, confirmed by engineering faces with drastically different appearance that elicited identical responses in single face cells. Our work suggests that other objects could be encoded by analogous metric coordinate systems. PAPERCLIP.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior investigations have provided evidence for neurons that encode another animal's familiarity or dominance.
“They have also provided evidence for neurons that encode another animals’ familiarity or dominance 12,13”
From Reduced sociability and social agency encoding in adult Shank3-mutant mice are restored through gene re-expression in real time (2021) · cites this paper as [13]
A brain-computational model of face recognition will have to explain the spatial clusters of face-selective units and the selectivities and invariances observed with fMRI and invasive recordings.
“A brain-computational model of face recognition16 will have to explain the spatial clusters of face-selective units and the selectivities and invariances observed with fMRI17,18 and invasive recordings14,19.”
From Cognitive computational neuroscience (2018) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409 · see citing works
Face recognition using eigenfaces
Turk, M., & Pentland, A. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1991.139758 · Cited by 5,232 · see citing works
Principal warps: thin-plate splines and the decomposition of deformations
Bookstein, F. L. (1989) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.24792 · Cited by 4,994 · see citing works
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. R. (1999) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/311535.311556 · Cited by 4,695 · see citing works
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309 · see citing works
Low-dimensional procedure for the characterization of human faces
Sirovich, L., & Kirby, M. J. L. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.000519 · Cited by 2,329 · see citing works
Invariant visual representation by single neurons in the human brain
Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005) · Nature · doi:10.1038/nature03687 · Cited by 1,988 · see citing works
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924 · see citing works
The importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · see citing works
Inferotemporal cortex and object vision
Tanaka, K. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.000545 · Cited by 1,402 · see citing works
A Unified Account of the Effects of Distinctiveness, Inversion, and Race in Face Recognition
Valentine, T. (1991) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749108400966 · Cited by 1,388 · see citing works
Identifying natural images from human brain activity
Kay, K., Naselaris, T., Prenger, R., & Gallant, J. L. (2008) · Nature · doi:10.1038/nature06713 · Cited by 1,272 · see citing works
A Cortical Region Consisting Entirely of Face-Selective Cells
Tsao, D. Y., Freiwald, W. A., Tootell, R. B. H., & Livingstone, M. S. (2006) · Science · doi:10.1126/science.1119983 · Cited by 1,131 · see citing works
Self-citationFreiwald, W. A., & Tsao, D. Y. (2010) · Science · doi:10.1126/science.1194908 · Cited by 678 · see citing works
Self-citationFace Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About
Sinha, P., Balas, B., Ostrovsky, Y., & Russell, R. (2006) · Proceedings of the IEEE · doi:10.1109/jproc.2006.884093 · Cited by 674 · see citing works
Comparing face patch systems in macaques and humans
Tsao, D. Y., Moeller, S., & Freiwald, W. A. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0809662105 · Cited by 661 · see citing works
Self-citationFaces and objects in macaque cerebral cortex
Tsao, D. Y., Freiwald, W. A., Knutsen, T., Mandeville, J. B., & Tootell, R. B. H. (2003) · Nature Neuroscience · doi:10.1038/nn1111 · Cited by 636 · see citing works
Self-citationInterpreting face images using active appearance models
Edwards, G. J., Taylor, C., & Cootes, T. F. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670965 · Cited by 489 · see citing works
Spike-triggered neural characterization
Schwartz, O., Pillow, J. W., Rust, N. C., & Simoncelli, E. P. (2006) · Journal of Vision · doi:10.1167/6.4.13 · Cited by 441 · see citing works
A face feature space in the macaque temporal lobe
Freiwald, W. A., Tsao, D. Y., & Livingstone, M. S. (2009) · Nature Neuroscience · doi:10.1038/nn.2363 · Cited by 441 · see citing works
Self-citationSize and position invariance of neuronal responses in monkey inferotemporal cortex
Ito, M., Tamura, H., Fujita, I., & Tanaka, K. (1995) · Journal of Neurophysiology · doi:10.1152/jn.1995.73.1.218 · Cited by 432 · see citing works
Patches with Links: A Unified System for Processing Faces in the Macaque Temporal Lobe
Moeller, S., Freiwald, W. A., & Tsao, D. Y. (2008) · Science · doi:10.1126/science.1157436 · Cited by 402 · see citing works
Self-citationUnderlying principles of visual shape selectivity in posterior inferotemporal cortex
Brincat, S. L., & Connor, C. E. (2004) · Nature Neuroscience · doi:10.1038/nn1278 · Cited by 385 · see citing works
Norm-based face encoding by single neurons in the monkey inferotemporal cortex
Leopold, D. A., Bondar, I. V., & Giese, M. A. (2006) · Nature · doi:10.1038/nature04951 · Cited by 379 · see citing works
Image Representations for Visual Learning
Beymer, D., & Poggio, T. (1996) · Science · doi:10.1126/science.272.5270.1905 · Cited by 272 · see citing works
Majaj, N. J., Hong, H., Solomon, E. A., & DiCarlo, J. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5181-14.2015 · Cited by 193 · see citing works
Anatomical Connections of the Functionally Defined “Face Patches” in the Macaque Monkey
Grimaldi, P., Saleem, K. S., & Tsao, D. Y. (2016) · Neuron · doi:10.1016/j.neuron.2016.05.009 · Cited by 140 · see citing works
Self-citationMedial Axis Shape Coding in Macaque Inferotemporal Cortex
Hung, C.-C., Carlson, E. T., & Connor, C. E. (2012) · Neuron · doi:10.1016/j.neuron.2012.04.029 · Cited by 139 · see citing works
Neural portraits of perception: Reconstructing face images from evoked brain activity
Cowen, A., Chun, M. M., & Kuhl, B. A. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.03.018 · Cited by 113 · see citing works
Sources of adaptation of inferior temporal cortical responses
Vogels, R. (2016) · Cortex · doi:10.1016/j.cortex.2015.08.024 · Cited by 97 · see citing works
Feature-based face representations and image reconstruction from behavioral and neural data
Nestor, A., Plaut, D. C., & Behrmann, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1514551112 · Cited by 63 · see citing works
MR-guided stereotactic navigation
Ohayon, S., & Tsao, D. Y. (2011) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2011.11.031 · Cited by 50 · see citing works
Self-citationCalculating Confidence Intervals for Prediction Error in Microarray Classification Using Resampling
Jiang, W., Varma, S., & Simon, R. M. (2008) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1322 · Cited by 44 · see citing works
Face Alignment by Local Deep Descriptor RegressionPreprint
Kumar, A., Ranjan, R., Patel, V. M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.07950 · Cited by 20 · see citing works
Comment on "Why does deep and cheap learning work so well?" [arXiv:1608.08225]Preprint
Schwab, D. J., & Mehta, P. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.03541 · Cited by 9 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Lin H, and Tegmark M (2016). Why does deep and cheap learning work so well? arXiv.
- Moeller S, Freiwald WA, and Tsao DY (2008a). A direct link between face-selective regions in temporal and prefrontal cortex. Paper presented at: Society for Neuroscience Conference (Washington D.C.).
Cited by
Sorted by citations
Navigating cognition: Spatial codes for human thinking
Bellmund, J. L. S., Gärdenfors, P., Moser, E. I., & Doeller, C. F. (2018) · Science · doi:10.1126/science.aat6766 · Cited by 643 · see citing works
Cognitive computational neuroscience
Kriegeskorte, N., & Douglas, P. K. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0210-5 · Cited by 535 · see citing works
Dehaene-Lambertz, G., Monzalvo, K., & Dehaene, S. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004103 · Cited by 367 · see citing works
The Border Between Seeing and Thinking
Block, N. (2023) · doi:10.1093/oso/9780197622223.001.0001 · Cited by 367 · see citing works
Ponce, C. R., Xiao, W., Schade, P. F., Hartmann, T. S., Kreiman, G., & Livingstone, M. S. (2019) · Cell · doi:10.1016/j.cell.2019.04.005 · Cited by 245 · see citing works
Neural tuning and representational geometry
Kriegeskorte, N., & Wei, X.-X. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00502-3 · Cited by 187 · see citing works
Higgins, I., Chang, L., Langston, V., Hassabis, D., Summerfield, C., Tsao, D., & Botvinick, M. (2021) · Nature Communications · doi:10.1038/s41467-021-26751-5 · Cited by 162 · see citing works
Self-citationShared neural coding for social hierarchy and reward value in primate amygdala
Munuera, J., Rigotti, M., & Salzman, C. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0082-8 · Cited by 126 · see citing works
Structure learning and the posterior parietal cortex
Summerfield, C., Luyckx, F., & Sheahan, H. (2020) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2019.101717 · Cited by 118 · see citing works
Mind Reading and Writing: The Future of Neurotechnology
Roelfsema, P. R., Denys, D., & Klink, P. C. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.04.001 · Cited by 107 · see citing works
Bonner, M. F., & Epstein, R. A. (2021) · Nature Communications · doi:10.1038/s41467-021-24368-2 · Cited by 96 · see citing works
Hesse, J. K., & Tsao, D. Y. (2020) · eLife · doi:10.7554/elife.58360 · Cited by 78 · see citing works
Self-citationIssa, E. B., Cadieu, C. F., & DiCarlo, J. J. (2018) · eLife · doi:10.7554/elife.42870 · Cited by 76 · see citing works
Abstract representations emerge naturally in neural networks trained to perform multiple tasks
Johnston, W. J., & Fusi, S. (2023) · Nature Communications · doi:10.1038/s41467-023-36583-0 · Cited by 89 · see citing works
How deep is the brain? The shallow brain hypothesis
Suzuki, M., Pennartz, C. M. A., & Aru, J. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00756-z · Cited by 79 · see citing works
Rainey, S., Martin, S., Christen, A., Mégevand, P., & Fourneret, E. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00218-0 · Cited by 65 · see citing works
Conscious perception as integrated information patterns in human electrocorticography
Haun, A. M., Oizumi, M., Kovach, C. K., Kawasaki, H., Oya, H., Howard, M. A., Adolphs, R., & Tsuchiya, N. (2017) · eneuro · doi:10.1523/eneuro.0085-17.2017 · Cited by 47 · see citing works
Apical amplification—a cellular mechanism of conscious perception?
Marvan, T., Polák, M., Bachmann, T., & Phillips, W. A. (2021) · Neuroscience of Consciousness · doi:10.1093/nc/niab036 · Cited by 45 · see citing works
Speech-accompanying gestures are not processed by the language-processing mechanisms
Jouravlev, O., Zheng, D., Balewski, Z., Le Arnz Pongos, A., Levan, Z., Goldin-Meadow, S., & Fedorenko, E. (2019) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2019.107132 · Cited by 39 · see citing works
Lee, D. K., Li, S. W., Bounni, F., Friedman, G., Jamali, M., Strahs, L., Zeliger, O., Gabrieli, P., Stankovich, M. A., Demaree, J., et al. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00888-4 · Cited by 33 · see citing works
- Authors
- Chang, L., & Tsao, D. Y.
- Affiliations
- California Institute of Technology (US)
- Corresponding author
- Chang, Le
- Tsao, Doris Y.
- Venue
- Cell
- Publisher
- Elsevier BV
- Published
- 2017-06
- Volume
- 169
- Issue
- 6
- Pages
- 1013-1028.e14
- ISSN
- 0092-8674 (ISSN-L)
- PMC
- PMC8088389
- OpenAlex
- W2621150162
- Cited by
- 644
- References
- 35
- Citations by year
- 2026: 45 · 2025: 61 · 2024: 51 · 2023: 75 · 2022: 73 · 2021: 76 · 2020: 115 · 2019: 53 · 2018: 74 · 2017: 21
- FWCI
- 41.26
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Cell (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion)
- Funders
- NIH (1R01EY019702)
- Howard Hughes Medical Institute
- Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech
- Swartz Foundation (# 2013-36)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Keywords
- face identity, face recognition (cognition), face patches, population coding, linear dimensionality reduction, macaque visual cortex, object recognition
- MeSH terms
- Facial Recognition, Animals, Humans, Macaca, Magnetic Resonance Imaging, Male, Models, Neurological, Neurons, Temporal Lobe
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
