Learning Race from Face: A Survey
Open accessThe DOI matches its Crossref record; 219 of the 232 references it cites were found in a registry or an index.
Abstract
Faces convey a wealth of social signals, including race, expression, identity, age and gender, all of which have attracted increasing attention from multi-disciplinary research, such as psychology, neuroscience, computer science, to name a few. Gleaned from recent advances in computer vision, computer graphics, and machine learning, computational intelligence based racial face analysis has been particularly popular due to its significant potential and broader impacts in extensive real-world applications, such as security and defense, surveillance, human computer interface (HCI), biometric-based identification, among others. These studies raise an important question: How implicit, non-declarative racial category can be conceptually modeled and quantitatively inferred from the face? Nevertheless, race classification is challenging due to its ambiguity and complexity depending on context and criteria. To address this challenge, recently, significant efforts have been reported toward race detection and categorization in the community. This survey provides a comprehensive and critical review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications. We first discuss race perception problem formulation and motivation, while highlighting the conceptual potentials of racial face processing. Next, taxonomy of feature representational models, algorithms, performance and racial databases are presented with systematic discussions within the unified learning scenario. Finally, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potentially important cross-cutting themes and research directions for the issue of learning race from face.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Items 201–219 of 219
Coutinho, J., & de Carvalho, A. F. P. (2008) · InTech eBooks · doi:10.5772/5869 · Cited by 8
Soft Biometrics for Video Surveillance
Fu, Y., Guo, G., & Huang, T. S. (2009) · CRC Press eBooks · doi:10.1201/9781439813300-c15 · Cited by 7
Race recognition from face images using Weber local descriptor
Muhammad, G., Hussain, M., Alenezy, F., Bebis, G. N., Mirza, A. M., & Aboalsamh, H. (2012) · Cited by 7
Can computers cope with human races?
Earnest, L. (1989) · Communications of the ACM · doi:10.1145/63342.63343 · Cited by 7 · cited 1×
“Therefore, simply assigning any logically complete, consistent, and determinable categorical labels to these mixed-racial faces will fail to make any kind of objective and substantial sense [242].”
BackgroundThe Recognition of Ethnic Groups based on Histological Skin Properties
Malskies, C. R., Eibenberger, E., & Angelopoulou, E. (2011) · Eurographics · doi:10.2312/pe/vmv/vmv11/353-360 · Cited by 7
Allison, M., & Kendrick, L. M. (2013) · The Florida AI Research Society · Cited by 7
Modeling the Other-race Advantage with PCA
Haque, A., & Cottrell, G. W. (2005) · eScholarship (California Digital Library) · Cited by 7
Demographics versus Biometric Automatic Interoperability
Marsico, M. D., Nappi, M., Riccio, D., & Wechsler, H. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41181-6_48 · Cited by 6 · cited 1×
“In particular, experimental results in [28], [29], [30], [31], [68], [169], [175], [220], [221], [222] have shown that both commercial and nontrainable face recognition algorithms consistently tend to perform lower on the same cohorts (females, African”
ResultPerformance Evaluation of Face Recognition Algorithms on the Asian Face Database, KFDB
Hwang, B. W., Byun, H., Roh, M.-C., & Lee, S.-W. (2003) · Lecture notes in computer science · Cited by 6
Hajj and Umrah Event Recognition DatasetsPreprint
Zawbaa, H. M., & Aly, S. A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2345 · Cited by 5
Ethnicity Classification Based on a Hierarchical Fusion
De, Z., Wang, Y., & Zhang, Z. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35136-5_36 · Cited by 4 · cited 1×
“introducing a cascaded multi-modal biometrics system involving a fusion of frontal face and lateral gait traits in [164].”
BackgroundA Novel Survey Based on Multiethnic Facial Semantic Web
Li, Z., Xiaodong, D., & Qingling, Z. (2013) · TELKOMNIKA Indonesian Journal of Electrical Engineering · doi:10.11591/telkomnika.v11i9.3251 · Cited by 4 · cited 1×
“[117], [166] have suggested distinct differences maybe exist even in the same ethnic group geographically, this has been supported by astrometric characteristics among three Chinese ethnic groups (sub-ethnic groups in Mongoloid”
BackgroundFuzzy Moments Method for Face Recognition in an Ethnic Database
Akbari, R., & Mozaffari, S. (2015) · Cited by 3
Determinants of the Own-Race Bias and neural correlates of own- and other-race face processing
Stahl, J. (2011) · Common Library Network (Der Gemeinsame Bibliotheksverbund) · Cited by 3
Gender and Race Identification by Man and Machine
Tariq, U., Hu, Y., & Huang, T. S. (2011) · Springer eBooks · doi:10.1007/978-3-642-22407-2_13 · Cited by 2 · cited 1×
“Empirical results from psychology [213] and computer vision [165], [214] have both confirmed silhouette’s”
BackgroundWhat gives a face its ethnicity?
Bulthoff, I. (2012) · Journal of Vision · doi:10.1167/12.9.1282 · Cited by 2
fMRI used to study race effect on face recognition
Love, R. (2001) · The Lancet · doi:10.1016/s0140-6736(01)05671-9 · Cited by 2
Mackal, M. C. (2009) · Cited by 1
Learning structurally discriminant features in 3D faces
Sukumar, S. R., Bozdogan, H., Page, D. L., Koschan, A. F., & Abidi, M. A. (2008) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2008.4712154 · Cited by 1 · cited 1×
“[160] have used facial anthropometric data to learn structurally discriminant features in 3D face on a racially diversified database.”
Method
Research works printed but not linked: 16
- britannica, 2012, Encyclopcedia Britannica Ultimate Reference Suite
- lu, 0, Ethnicity identification from face images, SPIE International Symposium on Defense and Security Biometric Technology for Human Identification
- ramanathan, 2009, Age progression in human faces: A survey, J Vis Lang Comput, 15, 3349
- anitha, 2010, A survey on facial expression databases, Int J Eng Sci Technol, 2, 5158
- enlow, 1990, Facial Growth
- coon, 1962, The Origins of Races
- fu, 2009, Intelligent Video Surveillance Systems and Technology
- jain, 0, Can soft biometric traits assist user recognition?, Proc Defense Security Symp Int Soc Opt Photon, 561
- evers, 1997, Human-computer interfaces Designing for culture
- 10.1207/s15516709cog2606_4 · doi:10.1207/s15516709cog2606_4 · Paper page
- lecun, 1995, Convolutional networks for images speech, time-series, The Handbook of Brain Theory and Neural Networks
- otoole, 0, Classifying faces by race and sex using an autoassociative memory trained for recognition, Proc 13th Annu Conf Cogn Sci Soc, 847
- tin, 2011, Race identification for face images, ACEEE Int J Inform Tech, 1
- 10.1037/0278-7393.22.6.1364 · doi:10.1037/0278-7393.22.6.1364 · Paper page
- brewer, 1998, Intergroup Relations
- ahmad, 2013, Arabic race classification of face images, International Journal of Computer Technology, 4, 234
Cited by
What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics
Dantcheva, A., Elia, P., & Ross, A. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2480381 · Cited by 435
“[55] which provides a comprehensive study on human and automated analysis of race classification.”
BackgroundA hybrid deep learning CNN–ELM for age and gender classification
Duan, M., Li, K., Yang, C., & Li, K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.08.062 · Cited by 258
“Recently, age and gender classification has received huge attenion, which provides direct and quickest way for obtaining implicit nd critical social information [15] .”
BackgroundScheuerman, M. K., Wade, K., Lustig, C., & Brubaker, J. R. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392866 · Cited by 193
“Like PPB, these different sources also drove the annotation process: As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [50, 54, 130, 131].”
BackgroundFace Recognition: Too Bias, or Not Too Bias?
Robinson, J. P., Livitz, G., Henon, Y., Qin, C., Fu, Y., & Timoner, S. J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00008 · Cited by 180
“This was then further refined by processing faces using ethnicity [12] and gender [20] classifiers.”
MethodMerler, M., Ratha, N., Feris, R. S., & Smith, J. R. (2019) · arXiv · doi:10.48550/arxiv.1901.10436 · Cited by 30
“As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [6,63–65].”
Background
Metadata
- Authors
- Fu, S., He, H., & Hou, Z.-G.
- Affiliations
- University of Rhode Island (US)
- Chinese Academy of Sciences (CN)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2014-12-01
- Volume
- 36
- Issue
- 12
- Pages
- 2483-2509
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W1993766142
- Cited by
- 178
- References
- 232
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 11 · 2023: 17 · 2022: 15 · 2021: 21 · 2020: 21 · 2019: 20 · 2018: 18 · 2017: 19 · 2016: 8 · 2015: 10 · 2014: 2
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- Funders
- US National Science Foundation (NSF) (1053717)
- Army Research Office (ARO) (W911NF-12-1-0378)
- NSF-DFG Collaborative Research on “Autonomous Learning” (CNS 1117314)
- National Natural Science Foundation of China (NSFC) (61225017, 61175076)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- racial classification, face recognition, race perception, biometric identification, surveillance, human-computer interaction
- MeSH terms
- Machine Learning, Anthropometry, Face, Female, Humans, Image Processing, Computer-Assisted, Male, Racial Groups, Biometric Identification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.