CoChat
Sign Up
Cited by 193Ethics and Social Impacts of AIFace recognition and analysisBiometric Identification and Security

How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis

Scheuerman, Morgan Klaus, Wade, Kandrea, Lustig, Caitlin, Brubaker, Jed R.·2020·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3392866
Open this paper in CoChat — free account

Abstract

Race and gender have long sociopolitical histories of classification in technical infrastructures-from the passport to social media. Facial analysis technologies are particularly pertinent to understanding how identity is operationalized in new technical systems. What facial analysis technologies can do is determined by the data available to train and evaluate them with. In this study, we specifically focus on this data by examining how race and gender are defined and annotated in image databases used for facial analysis. We found that the majority of image databases rarely contain underlying source material for how those identities are defined. Further, when they are annotated with race and gender information, database authors rarely describe the process of annotation. Instead, classifications of race and gender are portrayed as insignificant, indisputable, and apolitical. We discuss the limitations of these approaches given the sociohistorical nature of race and gender. We posit that the lack of critical engagement with this nature renders databases opaque and less trustworthy. We conclude by encouraging database authors to address both the histories of classification inherently embedded into race and gender, as well as their positionality in embedding such classifications.

Integrity signals

Registry

Crossref · 10.1145/3392866

Cites

112 of 112 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Other research has uncovered the ambiguity and uncertainty inherent in socially constructed labels, such as gender, accent, and content toxicity.

    “Other research has uncovered the ambiguity and uncertainty inherent in socially constructed labels, such as gender [85], accent [81], and content toxicity [1, 16].”

    From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [85]

  • Machine learning models, such as those offered by API providers, are prone to exhibiting various kinds of discriminatory behaviours, including different error rates across demographic groups, stereotyping, or minority groups under-representation.

    “It is well acknowledged that machine learning models, such as those ofered by the API providers, are prone to exhibiting various kinds of discriminatory behaviours, including diferent error rates across demographic groups [22,126,127], stereotyping [64,163] or minority groups under-representation [134,137].”

    From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [134]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Scheuerman, M. K., Wade, K., Lustig, C., & Brubaker, J. R.
Affiliations
  • University of Colorado Boulder (US)
  • University of Washington (US)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2020-05-28
Volume
4
Issue
CSCW1
Pages
1-35
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3392866
OpenAlex
W3029264758
Cited by
193
References
112
Citations by year
2026: 12 · 2025: 28 · 2024: 44 · 2023: 32 · 2022: 39 · 2021: 31 · 2020: 7
FWCI
37.14
Type
journal article
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
Keywords
facial analysis, image database, race, gender, identity classification, annotation practices, algorithmic bias, technological infrastructure, classification history, positionality, face recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review