Learning Race from Face: A Survey
Open accessThe DOI matches its Crossref record; 219 of the 232 references it cites were found in a registry or an index.
Abstract
Faces convey a wealth of social signals, including race, expression, identity, age and gender, all of which have attracted increasing attention from multi-disciplinary research, such as psychology, neuroscience, computer science, to name a few. Gleaned from recent advances in computer vision, computer graphics, and machine learning, computational intelligence based racial face analysis has been particularly popular due to its significant potential and broader impacts in extensive real-world applications, such as security and defense, surveillance, human computer interface (HCI), biometric-based identification, among others. These studies raise an important question: How implicit, non-declarative racial category can be conceptually modeled and quantitatively inferred from the face? Nevertheless, race classification is challenging due to its ambiguity and complexity depending on context and criteria. To address this challenge, recently, significant efforts have been reported toward race detection and categorization in the community. This survey provides a comprehensive and critical review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications. We first discuss race perception problem formulation and motivation, while highlighting the conceptual potentials of racial face processing. Next, taxonomy of feature representational models, algorithms, performance and racial databases are presented with systematic discussions within the unified learning scenario. Finally, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potentially important cross-cutting themes and research directions for the issue of learning race from face.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271 · cited 1×
“A promising direction for future research, therefore, is the development of recognition models that take visual attention models [236], [237], [238] and eye tracking data [232], [234] into account.”
BackgroundHe, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Self-citationComputational modelling of visual attention
Itti, L., & Koch, C. (2001) · Nature Reviews Neuroscience · doi:10.1038/35058500 · Cited by 4,733 · cited 1×
“A promising direction for future research, therefore, is the development of recognition models that take visual attention models [236], [237], [238] and eye tracking data [232], [234] into account.”
BackgroundUnderstanding face recognition
Bruce, V., & Young, A. (1986) · British Journal of Psychology · doi:10.1111/j.2044-8295.1986.tb02199.x · Cited by 3,912
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Overview of the Face Recognition Grand Challenge
Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.268 · Cited by 2,438
Emotion, cognition, and behavior
Dolan, R. J. (2002) · Science · doi:10.1126/science.1076358 · Cited by 1,979
High resolution of human evolutionary trees with polymorphic microsatellites
Bowcock, A. M., Ruiz-Linares, A., Tomfohrde, J., Minch, E., Kidd, J. R., & Cavalli-Sforza, L. L. (1994) · Nature · doi:10.1038/368455a0 · Cited by 1,850
(2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1 · Cited by 1,819
Research works printed but not linked: 16
- britannica, 2012, Encyclopcedia Britannica Ultimate Reference Suite
- lu, 0, Ethnicity identification from face images, SPIE International Symposium on Defense and Security Biometric Technology for Human Identification
- ramanathan, 2009, Age progression in human faces: A survey, J Vis Lang Comput, 15, 3349
- anitha, 2010, A survey on facial expression databases, Int J Eng Sci Technol, 2, 5158
- enlow, 1990, Facial Growth
- coon, 1962, The Origins of Races
- fu, 2009, Intelligent Video Surveillance Systems and Technology
- jain, 0, Can soft biometric traits assist user recognition?, Proc Defense Security Symp Int Soc Opt Photon, 561
- evers, 1997, Human-computer interfaces Designing for culture
- 10.1207/s15516709cog2606_4 · doi:10.1207/s15516709cog2606_4 · Paper page
- lecun, 1995, Convolutional networks for images speech, time-series, The Handbook of Brain Theory and Neural Networks
- otoole, 0, Classifying faces by race and sex using an autoassociative memory trained for recognition, Proc 13th Annu Conf Cogn Sci Soc, 847
- tin, 2011, Race identification for face images, ACEEE Int J Inform Tech, 1
- 10.1037/0278-7393.22.6.1364 · doi:10.1037/0278-7393.22.6.1364 · Paper page
- brewer, 1998, Intergroup Relations
- ahmad, 2013, Arabic race classification of face images, International Journal of Computer Technology, 4, 234
Cited by
What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics
Dantcheva, A., Elia, P., & Ross, A. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2480381 · Cited by 435
“[55] which provides a comprehensive study on human and automated analysis of race classification.”
BackgroundA hybrid deep learning CNN–ELM for age and gender classification
Duan, M., Li, K., Yang, C., & Li, K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.08.062 · Cited by 258
“Recently, age and gender classification has received huge attenion, which provides direct and quickest way for obtaining implicit nd critical social information [15] .”
BackgroundScheuerman, M. K., Wade, K., Lustig, C., & Brubaker, J. R. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392866 · Cited by 193
“Like PPB, these different sources also drove the annotation process: As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [50, 54, 130, 131].”
BackgroundFace Recognition: Too Bias, or Not Too Bias?
Robinson, J. P., Livitz, G., Henon, Y., Qin, C., Fu, Y., & Timoner, S. J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00008 · Cited by 180
“This was then further refined by processing faces using ethnicity [12] and gender [20] classifiers.”
MethodMerler, M., Ratha, N., Feris, R. S., & Smith, J. R. (2019) · arXiv · doi:10.48550/arxiv.1901.10436 · Cited by 30
“As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [6,63–65].”
Background
Metadata
- Authors
- Fu, S., He, H., & Hou, Z.-G.
- Affiliations
- University of Rhode Island (US)
- Chinese Academy of Sciences (CN)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2014-12-01
- Volume
- 36
- Issue
- 12
- Pages
- 2483-2509
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W1993766142
- Cited by
- 178
- References
- 232
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 11 · 2023: 17 · 2022: 15 · 2021: 21 · 2020: 21 · 2019: 20 · 2018: 18 · 2017: 19 · 2016: 8 · 2015: 10 · 2014: 2
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- Funders
- US National Science Foundation (NSF) (1053717)
- Army Research Office (ARO) (W911NF-12-1-0378)
- NSF-DFG Collaborative Research on “Autonomous Learning” (CNS 1117314)
- National Natural Science Foundation of China (NSFC) (61225017, 61175076)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- racial classification, face recognition, race perception, biometric identification, surveillance, human-computer interaction
- MeSH terms
- Machine Learning, Anthropometry, Face, Female, Humans, Image Processing, Computer-Assisted, Male, Racial Groups, Biometric Identification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning Race from Face: A Survey. https://app.cochat.ai/papers/10.1109/tpami.2014.2321570
- The paper
- Fu, S., He, H., & Hou, Z.-G. (2014). Learning Race from Face: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2014.2321570
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.