CoChat
Sign Up
Cited by 253Ethics and Social Impacts of AIFace recognition and analysisLGBTQ Health, Identity, and Policy

How Computers See Gender: An Evaluation of Gender Classification in Commercial Facial Analysis Services

Scheuerman, Morgan Klaus, Paul, Jacob M., Brubaker, Jed R.·2019·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3359246
Open this paper in CoChat — free account

Abstract

Investigations of facial analysis (FA) technologies-such as facial detection and facial recognition-have been central to discussions about Artificial Intelligence's (AI) impact on human beings. Research on automatic gender recognition, the classification of gender by FA technologies, has raised potential concerns around issues of racial and gender bias. In this study, we augment past work with empirical data by conducting a systematic analysis of how gender classification and gender labeling in computer vision services operate when faced with gender diversity. We sought to understand how gender is concretely conceptualized and encoded into commercial facial analysis and image labeling technologies available today. We then conducted a two-phase study: (1) a system analysis of ten commercial FA and image labeling services and (2) an evaluation of five services using a custom dataset of diverse genders using self-labeled Instagram images. Our analysis highlights how gender is codified into both classifiers and data standards. We found that FA services performed consistently worse on transgender individuals and were universally unable to classify non-binary genders. In contrast, image labeling often presented multiple gendered concepts. We also found that user perceptions about gender performance and identity contradict the way gender performance is encoded into the computer vision infrastructure. We discuss our findings from three perspectives of gender identity (self-identity, gender performativity, and demographic identity) and how these perspectives interact across three layers: the classification infrastructure, the third-party applications that make use of that infrastructure, and the individuals who interact with that software. We employ Bowker and Star's concepts of "torque" and "residuality" to further discuss the social implications of gender classification. We conclude by outlining opportunities for creating more inclusive classification infrastructures and datasets, as well as with implications for policy.

Integrity signals

Registry

Crossref · 10.1145/3359246

Cites

99 of 99 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AIaaS might contribute to replicating fairness issues at scale by propagating a model's intrinsic biases to the vast number of user applications powered by a given service.

    “This means that AIaaS might not only contribute to replicating fairness issues at scale by propagating model's intrinsic biases (or other issues) to the vast number of user applications powered by a given service (see previous work including [22,81,133,139]).”

    From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [133]

  • There is some work on fairness relating to AI services, though this tends to focus on either exposing or 'auditing' certain AI services, or devising tools, methods and interventions relevant for those building (rather than using) such services.

    “There is some work on fairness relating to AI services, though this tends to focus on either exposing or 'auditing' certain AI services [22,38,126,133], or devising tools, methods and interventions relevant for those building (rather than using) such services [12,96,106,128].”

    From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [133]

  • AI API services are often openly accessible to anyone, allowing potential users, researchers, and regulators to examine and validate them to some extent, for example by validating performance against different benchmarks and types of data.

    “Whereas AI API services are often openly accessible to anyone, which allows potential users as well as researchers and regulators to examine and validate these services to some extent (e.g. by validating their performance against diferent benchmarks and types of data [22,133])”

    From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [133]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Scheuerman, M. K., Paul, J. M., & Brubaker, J. R.
Affiliations
University of Colorado Boulder (US)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2019-11-07
Volume
3
Issue
CSCW
Pages
1-33
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3359246
OpenAlex
W2989168403
Cited by
252
References
99
Citations by year
2026: 36 · 2025: 51 · 2024: 34 · 2023: 36 · 2022: 34 · 2021: 39 · 2020: 20 · 2019: 2
FWCI
41.55
Type
journal article
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
Funders
National Science Foundation (1042678)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Social Sciences › Psychology › Social Psychology › LGBTQ Health, Identity, and Policy
Keywords
facial analysis, gender classification, transgender people, non-binary gender, image annotation, gender identity, gender performativity, algorithmic bias, Instagram images

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review