Learning Race from Face: A Survey
Open accessThe DOI matches its Crossref record; 219 of the 232 references it cites were found in a registry or an index.
Abstract
Faces convey a wealth of social signals, including race, expression, identity, age and gender, all of which have attracted increasing attention from multi-disciplinary research, such as psychology, neuroscience, computer science, to name a few. Gleaned from recent advances in computer vision, computer graphics, and machine learning, computational intelligence based racial face analysis has been particularly popular due to its significant potential and broader impacts in extensive real-world applications, such as security and defense, surveillance, human computer interface (HCI), biometric-based identification, among others. These studies raise an important question: How implicit, non-declarative racial category can be conceptually modeled and quantitatively inferred from the face? Nevertheless, race classification is challenging due to its ambiguity and complexity depending on context and criteria. To address this challenge, recently, significant efforts have been reported toward race detection and categorization in the community. This survey provides a comprehensive and critical review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications. We first discuss race perception problem formulation and motivation, while highlighting the conceptual potentials of racial face processing. Next, taxonomy of feature representational models, algorithms, performance and racial databases are presented with systematic discussions within the unified learning scenario. Finally, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potentially important cross-cutting themes and research directions for the issue of learning race from face.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271 · cited 1×
“A promising direction for future research, therefore, is the development of recognition models that take visual attention models [236], [237], [238] and eye tracking data [232], [234] into account.”
BackgroundHe, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Self-citationComputational modelling of visual attention
Itti, L., & Koch, C. (2001) · Nature Reviews Neuroscience · doi:10.1038/35058500 · Cited by 4,733 · cited 1×
“A promising direction for future research, therefore, is the development of recognition models that take visual attention models [236], [237], [238] and eye tracking data [232], [234] into account.”
BackgroundUnderstanding face recognition
Bruce, V., & Young, A. (1986) · British Journal of Psychology · doi:10.1111/j.2044-8295.1986.tb02199.x · Cited by 3,912
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Overview of the Face Recognition Grand Challenge
Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.268 · Cited by 2,438
Emotion, cognition, and behavior
Dolan, R. J. (2002) · Science · doi:10.1126/science.1076358 · Cited by 1,979
High resolution of human evolutionary trees with polymorphic microsatellites
Bowcock, A. M., Ruiz-Linares, A., Tomfohrde, J., Minch, E., Kidd, J. R., & Cavalli-Sforza, L. L. (1994) · Nature · doi:10.1038/368455a0 · Cited by 1,850
(2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1 · Cited by 1,819
Research works printed but not linked: 16
- britannica, 2012, Encyclopcedia Britannica Ultimate Reference Suite
- lu, 0, Ethnicity identification from face images, SPIE International Symposium on Defense and Security Biometric Technology for Human Identification
- ramanathan, 2009, Age progression in human faces: A survey, J Vis Lang Comput, 15, 3349
- anitha, 2010, A survey on facial expression databases, Int J Eng Sci Technol, 2, 5158
- enlow, 1990, Facial Growth
- coon, 1962, The Origins of Races
- fu, 2009, Intelligent Video Surveillance Systems and Technology
- jain, 0, Can soft biometric traits assist user recognition?, Proc Defense Security Symp Int Soc Opt Photon, 561
- evers, 1997, Human-computer interfaces Designing for culture
- 10.1207/s15516709cog2606_4 · doi:10.1207/s15516709cog2606_4 · Paper page
- lecun, 1995, Convolutional networks for images speech, time-series, The Handbook of Brain Theory and Neural Networks
- otoole, 0, Classifying faces by race and sex using an autoassociative memory trained for recognition, Proc 13th Annu Conf Cogn Sci Soc, 847
- tin, 2011, Race identification for face images, ACEEE Int J Inform Tech, 1
- 10.1037/0278-7393.22.6.1364 · doi:10.1037/0278-7393.22.6.1364 · Paper page
- brewer, 1998, Intergroup Relations
- ahmad, 2013, Arabic race classification of face images, International Journal of Computer Technology, 4, 234
Cited by
What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics
Dantcheva, A., Elia, P., & Ross, A. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2480381 · Cited by 435
“[55] which provides a comprehensive study on human and automated analysis of race classification.”
BackgroundA hybrid deep learning CNN–ELM for age and gender classification
Duan, M., Li, K., Yang, C., & Li, K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.08.062 · Cited by 258
“Recently, age and gender classification has received huge attenion, which provides direct and quickest way for obtaining implicit nd critical social information [15] .”
BackgroundScheuerman, M. K., Wade, K., Lustig, C., & Brubaker, J. R. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392866 · Cited by 193
“Like PPB, these different sources also drove the annotation process: As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [50, 54, 130, 131].”
BackgroundFace Recognition: Too Bias, or Not Too Bias?
Robinson, J. P., Livitz, G., Henon, Y., Qin, C., Fu, Y., & Timoner, S. J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00008 · Cited by 180
“This was then further refined by processing faces using ethnicity [12] and gender [20] classifiers.”
MethodMerler, M., Ratha, N., Feris, R. S., & Smith, J. R. (2019) · arXiv · doi:10.48550/arxiv.1901.10436 · Cited by 30
“As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [6,63–65].”
BackgroundDiscovering Fair Representations in the Data Domain
Quadrianto, N., Sharmanska, V., & Thomas, O. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00842 · Cited by 172
“By using the photograph as is, a discrimination issue might arise, as photographs with faces could reveal certain protected characteristics, such as gender, race, or age (e.g. [14, 5, 4, 29]).”
An Ensemble CNN2ELM for Age Estimation
Duan, M., Li, K., & Li, K. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2766583 · Cited by 111
“Recently, age and gender classification has received a large amount of attention, because it provides a direct and quick method for obtaining implicit and critical social information [41].”
MethodNixon, M. S., Correia, P. L., Nasrollahi, K., Moeslund, T. B., Hadid, A., & Tistarelli, M. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2015.08.006 · Cited by 119
A Survey on Interdependent Privacy
Humbert, M., Trubert, B., & Huguenin, K. (2019) · ACM Computing Surveys · doi:10.1145/3360498 · Cited by 118
“…attributes correlated with – thus inferable from – an individual’s image or speech (e.g., gender and age [Levi and Hassncer 2015], ethnicity [Fu et al. 2014], emotion, activity), and the fact that the individuals featured in the media are together, which reveals some information about their…”
BackgroundIntelligent deep learning based ethnicity recognition and classification using facial images
Sunitha, G., Geetha, K., Neelakandan, S., Pundir, A. K. S., Hemalatha, S., & Kumar, V. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104404 · Cited by 91
Auto-essentialization: Gender in automated facial analysis as extended colonial project
Scheuerman, M. K., Pape, M., & Hanna, A. (2021) · Big Data & Society · doi:10.1177/20539517211053712 · Cited by 75
Demographic Analysis from Biometric Data: Achievements, Challenges, and New Frontiers
Sun, Y., Zhang, M., Sun, Z., & Tan, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2669035 · Cited by 75
“Owing to these emerging applications, there has been an increasing interest in this topic, resulting in a large number of attempts and initiatives in both scientific and industrial communities: collection of specific biometric databases [6], [7], [8], [9], [10]; numerous research work on demographic estimation from va…”
BackgroundReview on the effects of age, gender, and race demographics on automatic face recognition
Abdurrahim, S. H., Samad, S. A., & Huddin, A. B. (2017) · The Visual Computer · doi:10.1007/s00371-017-1428-z · Cited by 69
AI and recruiting software: Ethical and legal implications
Fernández-Martínez, C., & Fernández, A. (2020) · Paladyn, Journal of Behavioral Robotics · doi:10.1515/pjbr-2020-0030 · Cited by 68
Age from Faces in the Deep Learning Revolution
Carletti, V., Greco, A., Percannella, G., & Vento, M. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2910522 · Cited by 73
“Survey Topic [5] Fu et al. 2010 Age estimation [11] Fu et al. 2014 Ethnicity recognition [12] Zafeiriou et al. 2015 Face detection [13] Ng et al. 2015 Gender recognition [14] Sarivanidi et al. 2015 Affect analysis [6] Dhimar et al. 2016 Age estimation [9] Dantcheva et al. 2016 Soft biometrics [8]…”
BackgroundEthical AI in facial expression analysis: racial bias
Sham, A. H., Aktas, K., Rizhinashvili, D., Kuklyanov, D., Alisinanoğlu, F., Ofodile, I., Ozcinar, C., & Anbarjafari, G. (2022) · Signal Image and Video Processing · doi:10.1007/s11760-022-02246-8 · Cited by 95
“The authors in [18] also mentioned the different databases that are commonly related to racial face such as: CAS-PEAL, IFDB, Texas 3DFRD, KFDB, JAFEE, CAFE, FGRC 2.”
BackgroundMethodInfluentialWhat We Teach About Race and Gender: Representation in Images and Text of Children’s Books
Adukia, A., Eble, A., Harrison, E., Runesha, H. B., & Szasz, T. (2023) · The Quarterly Journal of Economics · doi:10.1093/qje/qjad028 · Cited by 128
Deep Learning for Face Recognition: Pride or Prejudiced?Preprint
Nagpal, S., Singh, M., Singh, R., & Vatsa, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01219 · Cited by 114
“The own-race bias in humans has been well established, where we are able to recognize people of our own race more easily as compared to individuals of other race [26, 14].”
BackgroundA Novel Multi-task Tensor Correlation Neural Network for Facial Attribute Prediction
Duan, M., Li, K., Li, K., & Tian, Q. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3418285 · Cited by 62
“, prisoner tracking, pedestrian identification; (ii) face retrieval [14, 45, 48, 60]; and (iii) social media [46, 47], e.”
BackgroundRace Recognition Using Deep Convolutional Neural Networks
Vo, T., Nguyen, T., & Le, C. T. (2018) · Symmetry · doi:10.3390/sym10110564 · Cited by 59
DemogPairs: Quantifying the Impact of Demographic Imbalance in Deep Face Recognition
Hupont, I., & Fernández, C. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756625 · Cited by 80
“Consequently, it is of utmost importance to train with demographically balanced sets [9].”
BackgroundFace Recognition Based on Deep Learning and FPGA for Ethnicity Identification
AlBdairi, A. J. A., Xiao, Z., Alkhayyat, A., Humaidi, A. J., Fadhel, M. A., Taher, B. H., Alzubaidi, L., Santamaría, J., & Al-Shamma, O. (2022) · Applied Sciences · doi:10.3390/app12052605 · Cited by 54
Wang, Y., Li, Y., & Luo, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v10i1.14783 · Cited by 52
Computational Media Intelligence: Human-Centered Machine Analysis of Media
Somandepalli, K., Guha, T., Martínez, V. R., Kumar, N., Adam, H., & Narayanan, S. S. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3047978 · Cited by 62
“, gender [16], [17], race (see survey in [18]) and age [19], [20] have been widely studied in the speech processing and computer vision community, but as isolated topics, notably to ensure robust performance of algorithms e.”
BackgroundHOG-assisted deep feature learning for pedestrian gender recognition
Cai, L., Zhu, J., Zeng, H., Chen, J., Cai, C., & Ma, K. (2017) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2017.09.003 · Cited by 51
Truth from the machine: artificial intelligence and the materialization of identity
Keyes, O., Hitzig, Z., & Blell, M. (2021) · Interdisciplinary Science Reviews · doi:10.1080/03080188.2020.1840224 · Cited by 51
“The same is true of other e ff orts to infer identity from the face, in domains from disability and gender to race (Hashemi et al. 2012; Bautista et al. 2015; Fu, He, and Hou 2014).”
BackgroundCross-Modal Face Matching: Beyond Viewed Sketches
Ouyang, S., Hospedales, T. M., Song, Y.-Z., & Li, X. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16808-1_15 · Cited by 52
Speech Signal and Facial Image Processing for Obstructive Sleep Apnea Assessment
Espinoza-Cuadros, F., Pozo, R. F., Toledano, D. T., Alcázar-Ramírez, J., López-Gonzalo, E., & Gómez, L. A. H. (2015) · Computational and Mathematical Methods in Medicine · doi:10.1155/2015/489761 · Cited by 49
“On the other side, we believe that beyond simple measurements from facial landmarks the extraction of richer information related to facial shapes and textures using more powerful image processing techniques [47–49] can provide new insights into the OSA craniofacial phenotype and even characterize the sleepy facial app…”
BackgroundFusion of Domain-Specific and Trainable Features for Gender Recognition From Face Images
Azzopardi, G., Greco, A., Saggese, A., & Vento, M. (2018) · IEEE Access · doi:10.1109/access.2018.2823378 · Cited by 47
“For instance, by just looking at faces, humans recognize the gender and the ethnicity [1], [2], estimate the age [3], deduce the emotions and the state of mind [4], [5], determine if the person has a familiar face or is a stranger [6], and verify or recognize the identity of the individual [7]–[9].”
BackgroundOne Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision
Khan, Z., & Fu, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445920 · Cited by 46
Understanding Unequal Gender Classification Accuracy from Face Images
Muthukumar, V., Pedapati, T., Ratha, N., Sattigeri, P., Wu, C.-W., Kingsbury, B. E. D., Kumar, A., Thomas, S., Mojsilović, A., & Varshney, K. R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00099 · Cited by 42
“(Note that gender classification is a distinct task from race classification [15].)”
BackgroundHan, T.-Y., Bi, J.-W., Wei, Z., & Yao, Y. (2024) · Tourism Management · doi:10.1016/j.tourman.2024.104884 · Cited by 43
Benchmarking deep network architectures for ethnicity recognition using a new large face dataset
Greco, A., Percannella, G., Vento, M., & Vigilante, V. (2020) · Machine Vision and Applications · doi:10.1007/s00138-020-01123-z · Cited by 51
“It has been scien-tifically established that in face analysis tasks the humans perform significantly better when dealing with faces of people of their own ethnicity than with individuals belonging to other ethnicities; for this reason, this effect is also called own race bias [16].”
BackgroundInfluentialDuan, M., Li, K., Ouyang, A., Win, K. N., Li, K., & Tian, Q. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3379449 · Cited by 38
“Age classification has got huge amount of attention, because it supports an efficient approach for achieving implicit social information [16].”
BackgroundColor-Theoretic Experiments to Understand Unequal Gender Classification Accuracy From Face Images
Muthukumar, V., Pedapati, T., Ratha, N., Sattigeri, P., Wu, C.-W., Kingsbury, B. E. D., Kumar, A., Thomas, S., Mojsilović, A., & Varshney, K. R. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00282 · Cited by 45
“(Note that gender classifi-cation is a distinct task from race classification [16].)”
BackgroundA survey on facial soft biometrics for video surveillance and forensic applications
Becerra-Riera, F., Morales-González, A., & Méndez-Vázquez, H. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09689-5 · Cited by 45
“Ethnicity, however, refers to cultural factors that include nationality, regional culture and language (Fu et al. 2014).”
BackgroundMethodInfluentialFine-grained classification of pedestrians in video: Benchmark and state of the art
Hall, D., & Perona, P. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299187 · Cited by 35
Who Gets Started on Kickstarter? Racial Disparities in Crowdfunding SuccessPreprint
Rhue, L. A., & Clark, J. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2837042 · Cited by 35
Learned Features are Better for Ethnicity Classification
Anwar, I., & Islam, N. U. (2017) · Cybernetics and Information Technologies · doi:10.1515/cait-2017-0036 · Cited by 35
“These discriminative features can be differentiated in three categories [1], namely: chromatic/skin tone, local features and”
BackgroundBalancing Biases and Preserving Privacy on Balanced Faces in the Wild
Robinson, J. P., Qin, C., Henon, Y., Timoner, S. J., & Fu, Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3282837 · Cited by 39
“Then, the list was further refined by processing the corresponding faces with ethnicity [73] and gender [74] classifiers.”
MethodModeling of facial aging and kinship: A survey
Georgopoulos, M., Panagakis, Y., & Pantić, M. (2018) · Image and Vision Computing · doi:10.1016/j.imavis.2018.05.003 · Cited by 35
Do They All Look the Same? Deciphering Chinese, Japanese and Koreans by Fine-Grained Deep Learning
Wang, Y., Feng, Y., Liao, H., Luo, J., & Xu, X. (2018) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2018.00015 · Cited by 39
Hybrid Supervised Deep Learning for Ethnicity Classification using Face Images
Heng, Z., Manandhar, D., & Yap, K. (2018) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2018.8351370 · Cited by 31
“[6] conducted a thorough survey on some state-of-art advances in face-race perception, principles, algorithms and applications, as well as some feature representation models.”
BackgroundExpression of Concern: Facial feature discovery for ethnicity recognition
Wang, C., Zhang, Q., Liu, W., Liu, Y., & Miao, L. (2018) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1278 · Cited by 31
“The analysis of race, nation, and ethnical groups based on facial images is a popular topic recently in face recognition community (Fu, He, & Hou, 2014).”
Automatic facial attribute analysis via adaptive sparse representation of random patches
Mery, D., & Bowyer, K. W. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2015.05.005 · Cited by 31
“, gender [39], race [15], age [16], etc.”
BackgroundStudy of assessment of cognitive ability of human brain using deep learning
Sinha, G. R. (2017) · International Journal of Information Technology · doi:10.1007/s41870-017-0025-8 · Cited by 32
“The concept of race and face provides an interdisciplinary approach of research and development activities such as face databases classification, feature representation and classification [19].”
BackgroundWho Are You and What Are You Selling? Creator-Based and Product-Based Racial Cues in Crowdfunding
Rhue, L. A., & Clark, J. (2022) · MIS Quarterly · doi:10.25300/misq/2022/15214 · Cited by 31
The “Criminality From Face” Illusion
Bowyer, K. W., King, M. C., Scheirer, W. J., & Vangara, K. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.3032321 · Cited by 32
“Facial analytics can predict, with some reasonable accuracy, things, such as age [10], gender [6], race [9], facial expression/emotion [25], body mass index [5], and certain types of health conditions [29].”
BackgroundRecognizing Human Races through Machine Learning—A Multi-Network, Multi-Features Study
Darabant, A. S., Borza, D., & Danescu, R. (2021) · Mathematics · doi:10.3390/math9020195 · Cited by 24
“A detailed survey on racial and ethnical classification from facial images is presented in [3].”
BackgroundInfluentialAutomatic Ethnicity Classification from Middle Part of the Face Using Convolutional Neural Networks
Belcar, D., Grd, P., & Tomičić, I. (2022) · Informatics · doi:10.3390/informatics9010018 · Cited by 32
“[7] distinguish between four main methods of feature representation: chromatic representation, global feature representation, local feature descriptor representation and other representations.”
MethodDiscovering Fair Representations in the Data DomainPreprint
Quadrianto, N., Sharmanska, V., & Thomas, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06755 · Cited by 37
Rixen, J. O., Colley, M., Askari, A., Gugenheimer, J., & Rukzio, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502140 · Cited by 29
“AI also has started to outperform humans in a multitude of other areas like estimating a person’s age [19], sexual orientation, or race [20] through visual clues [63].”
BackgroundMethods to Establish Race or Ethnicity of Twitter Users: Scoping Review
Golder, S., Stevens, R. C., O’Connor, K., James, R., & Gonzalez‐Hernandez, G. (2022) · Journal of Medical Internet Research · doi:10.2196/35788 · Cited by 30
“Human annotation may introduce additional bias, and studies have found systematic biases in the classification of people into racial or ethnic groups based on photographs [207,208].”
BackgroundCraig, B. M., & Lee, A. J. (2020) · Adaptive Human Behavior and Physiology · doi:10.1007/s40750-020-00141-5 · Cited by 24
Emotion analysis from faces for social robotics
Greco, A., Roberto, A., Saggese, A., Vento, M., & Vigilante, V. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914039 · Cited by 23
“As for the second point, among the dynamic elements of interest for a social robot in order to properly interact with a person, we can mention the analysis of the actions or the gestures performed by the persons [4], [5], or the analysis of the face, aimed at estimating the gender [6], [7], the age [8], the ethnicity…”
BackgroundGender recognition in the wild: a robustness evaluation over corrupted images
Greco, A., Saggese, A., Vento, M., & Vigilante, V. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02750-0 · Cited by 20
Detection of Emotion Using Multi-Block Deep Learning in a Self-Management Interview App
Shin, D. H., Chung, K., & Park, R. C. (2019) · Applied Sciences · doi:10.3390/app9224830 · Cited by 22
“The AdaBoost algorithm is a technology often used for face detection, and is a method for creating strong criteria for selection by combining weak criteria, which has advantages [19,20].”
MethodWhat We Teach About Race and Gender: Representation in Images and Text of Children’s Books
Adukia, A., Eble, A., Harrison, E., Runesha, H. B., & Szasz, T. (2021) · National Bureau of Economic Research · doi:10.3386/w29123 · Cited by 30
Review of Demographic Fairness in Face Recognition
Kotwal, K., & Marcel, S. (2025) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2025.3601217 · Cited by 53
“The corresponding demographic labels were generated using ethnicity [90] and gender [91] classifiers, followed by manual validation.”
The reanimation of pseudoscience in machine learning and its ethical repercussions
Andrews, M., Smart, A., & Birhane, A. (2024) · Patterns · doi:10.1016/j.patter.2024.101027 · Cited by 18
Facecraft: Race Reification in Psychological Research With Faces
Martinez, J. E. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231194953 · Cited by 25
“…race given the combined levels of analysis; see, e.g., Kaiser Trujillo et al., 2022; Martínez Mateo et al., 2012) and that originate from different disciplines (e.g., development of algorithms within computer and data science that locate race in the face: Fu et al., 2014; Gabani et al., 2020).”
Modeling User Characteristics Associated with Interdependent Privacy Perceptions on Social Media
Amon, M. J., Necaise, A., Kartvelishvili, N., Williams, A., Solihin, Y., & Kapadia, A. C. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3577014 · Cited by 25
What We Teach About Race and Gender: Representation in Images and Text of Children’s BooksPreprint
Adukia, A., Eble, A., Harrison, E., Runesha, H. B., & Szasz, T. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3825080 · Cited by 23
“We expand and develop tools for image analysis, pairing them with tools from text analysis used in recent work by Caliskan, Bryson and Narayanan (2017), Garg et al. (2018), and Kozlowski, Taddy and Evans (2019).”
BackgroundMethodJoint gender, ethnicity and age estimation from 3D faces
Xia, B., Amor, B. B., & Daoudi, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.06.004 · Cited by 15
“Since the 90s of the previous century, several approaches have been proposed to solve the problem of image-based automatic facial Soft-Biometric Traits recognition, e.g. gender [22,26,45], ethnicity [17,26] and age [18,22,26,48].”
BackgroundIn-Group Bias in Deep Learning-Based Face Recognition Models Due to Ethnicity and Age
Nagpal, S., Singh, M., Singh, R., Vatsa, M., & Ratha, N. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3241010 · Cited by 20
“The in-group effect in humans has been well established, particularly for ethnicity, where we are able to recognize people of our own ethnicity more easily as compared to individuals of another ethnicity [5], [40].”
BackgroundA Novel Race Classification Method Based on Periocular Features Fusion
Chen, H., Gao, M., Ricanek, K., Xu, W.-L., & Fang, B. (2017) · International journal of pattern recognition and artificial intelligence · doi:10.1142/s0218001417500264 · Cited by 12
Improving Smiling Detection with Race and Gender Diversity
Ryu, H. J., Mitchell, M., & Adam, H. (2017) · Cited by 18
Surveying Racial Bias in Facial Recognition: Balancing Datasets and Algorithmic Enhancements
Sumsion, A. W., Torrie, S. A., Lee, D.-J., & Sun, Z. (2024) · Electronics · doi:10.3390/electronics13122317 · Cited by 23
Benítez-García, G., Nakamura, T., & Kaneko, M. (2018) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2017mvp0025 · Cited by 14
“HSV Quantization (HSVQ) is based on the fact that all possible skin tones of di ff erent human races are part of a subset color from elements of HSV [20].”
BackgroundConsidering Race a Problem of Transfer Learning
Khan, A., & Mahmoud, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00022 · Cited by 12
Zhang, X., Wang, J., Lane, J. M., Xu, X., & Sörensen, S. (2023) · Journal of Medical Internet Research · doi:10.2196/51089 · Cited by 12
“Most of this research, however, has focused on correctly identifying individual faces rather than categorizing them by race, and although there has been some discussion of the inaccuracy of ethnicity identification, categorization by race appears to be less controversial [43].”
Measuring diversity in Hollywood through the large-scale computational analysis of film
Bamman, D., Samberg, R., So, R. J., & Zhou, N. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2409770121 · Cited by 18
Attamimi, M., Ando, Y., Nakamura, T., Nagai, T., Mochihashi, D., Kobayashi, I., & Asoh, H. (2016) · Advanced Robotics · doi:10.1080/01691864.2016.1172507 · Cited by 12
Voting with Feet: Who are Leaving Hillary Clinton and Donald Trump
Wang, Y., Feng, Y., Luo, J., & Zhang, X. (2016) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism.2016.0022 · Cited by 12
“[5] provides a comprehensive survey of race classification based on facial features.”
BackgroundEnsuring privacy in face recognition: a survey on data generation, inference and storage
Sun, Z., & Liu, Z. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-06987-2 · Cited by 20
“Facial data are inherently sensitive as biometric information, encompassing personal details such as age, gender, and racial characteristics [25–27].”
3D face analysis for demographic biometrics
Tokola, R., Mikkilineni, A. K., & Boehnen, C. B. (2015) · International Conference on Biometrics · doi:10.1109/icb.2015.7139052 · Cited by 11
“[7] survey race classification.”
Large-scale geo-facial image analysis
Islam, M. T., Greenwell, C., Souvenir, R., & Jacobs, N. (2015) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-015-0070-9 · Cited by 12
“, “Where is this person probably from?” [50]), or school children (e.”
BackgroundRecognition of Facial Attributes Using Adaptive Sparse Representations of Random Patches
Mery, D., & Bowyer, K. W. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16181-5_59 · Cited by 14
“For human beings it is very difficult to distinguish a race, because it depends on how people self identify 6 , however, in our paper, the term ‘race’ –as in [8]– refers to a person’s physical appearance rather than sociological and cultural concepts like ethnicity.”
BackgroundTowards large-scale face-based race classification on spark framework
Mezzoudj, S., Behloul, A., & Seghir, R. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7672-7 · Cited by 13
Gender identification from facial images using global features
Mohamed, S. A. E. H., Nour, N., & Viriri, S. (2018) · Conference on Information Communications Technology and Society (ICTAS) · doi:10.1109/ictas.2018.8368761 · Cited by 10
Katti, H., & Arun, S. P. (2019) · Journal of Vision · doi:10.1167/19.7.1 · Cited by 14
Learning Fair Face Representation With Progressive Cross TransformerPreprint
Li, Y., Sun, Y., Cui, Z., Shan, S., & Yang, J. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04983 · Cited by 14
“This phenomenon is in line with the observations in [54], [55] that facial texture, facial features (e.g., eyes, nose, mouth) and facial contour are the major determinants for different racial cohorts.”
BackgroundResultA Survey on Bias in Machine Learning Research
Mikołajczyk, A., & Grochowski, M. (2025) · IEEE Access · doi:10.1109/access.2025.3649998 · Cited by 12
“Furthermore, we briefly presented methods for detecting and mitigating bias, including fairness metrics, debiasing techniques, and explainability methods.”
Facial Anthropometric Measurements and Photographs — An Interdisciplinary Study
Dhaliwal, J., Wagner, J., Leong, S. L., & Lim, C. H. (2020) · IEEE Access · doi:10.1109/access.2020.3028694 · Cited by 9
“It is worth-while to mention that reference [25] is a recent survey that provides a detailed review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications.”
BackgroundOn Classifying Facial Races with Partial Occlusions and Pose Variations
Alafif, T., Hailat, Z., Aslan, M. S., & Chen, X. (2017) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2017.00-82 · Cited by 10
“A number of recent surveys for iris and facial based race classification have been reported in [4] [7] [8] [9].”
BackgroundLabeled-Robust Regression: Simultaneous Data Recovery and Classification
Zeng, D., Wu, Z., Ding, C. H. Q., Ren, Z., Yang, Q., & Xie, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3026101 · Cited by 9
Facial Soft Biometrics Detection on Low Power Devices
Vasileiadis, M., Stavropoulos, G., & Tzovaras, D. K. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00285 · Cited by 9
“Early facial soft biometric-based approaches undertook the extraction of individual soft biometric traits, with the majority of the research work focusing on age [35, 18, 65], gender [59, 71] and race [22] estimation from face images.”
BackgroundOn Combining Face Local Appearance and Geometrical Features for Race Classification
Becerra-Riera, F., Méndez-Llanes, N., Morales-González, A., Méndez-Vázquez, H., & Massimo Tistarelli, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-13469-3_66 · Cited by 8
A multi-color based features from facial images for automatic ethnicity identification model
Osman, M. Z., Maarof, M. A., Rohani, M. F., Sjarif, N. N. A., & Zulkifli, N. S. A. (2020) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v18.i3.pp1383-1390 · Cited by 8
“Ethnicity identification is the key demographic attribute, and it plays an important role in many applications [12-14].”
Lionnie, R., Apriono, C., & Gunawan, D. (2022) · Algorithms · doi:10.3390/a15060208 · Cited by 8
Exploring the Effects of Head-Mounted Augmented Reality on Helping Behaviour
Rixen, J. O., Belz, J. H., Meinhardt, L.-M., Gugenheimer, J., & Rukzio, E. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627969 · Cited by 9
“WithAIstartingtooutperformhumansinamultitudeof areas(e.g.,detectingemotionalstatethroughmicroexpressions[54], person’sage[19],sexualorientationorrace[26]throughvisualclues [78]),thecombinationofARandAIcouldrevealnewinformation abouttheuser’ssituationtothem.”
Inclusive AI in Recruiting. Multi-agent Systems Architecture for Ethical and Legal Auditing
Fernández, C., & Fernández, A. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-24299-2_30 · Cited by 7
Beyond Vanilla Convolution: Random Pixel Difference Convolution for Face Perception
Liu, W., Su, Z., & Liu, L. (2021) · IEEE Access · doi:10.1109/access.2021.3117955 · Cited by 7
“Consequently, it is reasonable that multi-modality approaches shall surpass either mode alone [18].”
BackgroundMulti-ethnical Chinese facial characterization and analysis
Wang, C., Zhang, Q., Duan, X., & Gan, J. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6018-1 · Cited by 7
“The authors of, [11] stated that the recognition of racial and ethnic groups is a top priority when human faces are seen and recognized.”
On the Ethnic Classification of Pakistani Face using Deep Learning
Jilani, S. K., Ugail, H., Bukar, A. M., & Logan, A. J. (2019) · International Conference on Cyberworlds (CW) · doi:10.1109/cw.2019.00039 · Cited by 7
“The most comprehensive review on learning race from a face was written by Fu et al., [7].”
BackgroundBanati, U., Prakash, V., Verma, R., & Srivast, S. (2022) · Research Square · doi:10.21203/rs.3.rs-1174842/v1 · Cited by 7
Tian, T. Y., & Smith, E. B. (2024) · Administrative Science Quarterly · doi:10.1177/00018392241247744 · Cited by 6
“We followed the former convention and coded demographic variables on gender and race visually based on evidence of automatic racial classification based on facial perception (Telles, 2002; Saperstein, 2006; Brown, Hitlin, and Elder, 2007; Campbell and Troyer, 2007; Fu, He, and Hou, 2014).”
Label propagation approach for predicting missing biographic labels in face‐based biometric records
Swearingen, T., & Ross, A. A. (2017) · IET Biometrics · doi:10.1049/iet-bmt.2017.0117 · Cited by 6
Bagchi, C., Amali, D., & Dinakaran, M. (2018) · Advances in Intelligent Systems and Computing · doi:10.1007/978-981-13-3329-3_12 · Cited by 7
Performance Assessment of Face Analysis Algorithms with Occluded Faces
Greco, A., Saggese, A., Vento, M., & Vigilante, V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68790-8_37 · Cited by 6
Metadata
- Authors
- Fu, S., He, H., & Hou, Z.-G.
- Affiliations
- University of Rhode Island (US)
- Chinese Academy of Sciences (CN)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2014-12-01
- Volume
- 36
- Issue
- 12
- Pages
- 2483-2509
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W1993766142
- Cited by
- 178
- References
- 232
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 11 · 2023: 17 · 2022: 15 · 2021: 21 · 2020: 21 · 2019: 20 · 2018: 18 · 2017: 19 · 2016: 8 · 2015: 10 · 2014: 2
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- Funders
- US National Science Foundation (NSF) (1053717)
- Army Research Office (ARO) (W911NF-12-1-0378)
- NSF-DFG Collaborative Research on “Autonomous Learning” (CNS 1117314)
- National Natural Science Foundation of China (NSFC) (61225017, 61175076)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- racial classification, face recognition, race perception, biometric identification, surveillance, human-computer interaction
- MeSH terms
- Machine Learning, Anthropometry, Face, Female, Humans, Image Processing, Computer-Assisted, Male, Racial Groups, Biometric Identification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.