CoChat
Sign Up
Cited by 81Ethics and Social Impacts of AICybernetics and Technology in SocietyArtificial Intelligence in Law

To Live in Their Utopia: Why Algorithmic Systems Create Absurd Outcomes

Alkhatib, Ali·2021·Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3411764.3445740
Open this paper in CoChat — free account

Abstract

The promise AI’s proponents have made for decades is one in which our needs are predicted, anticipated, and met - often before we even realize it. Instead, algorithmic systems, particularly AIs trained on large datasets and deployed to massive scales, seem to keep making the wrong decisions, causing harm and rewarding absurd outcomes. Attempts to make sense of why AIs make wrong calls in the moment explain the instances of errors, but how the environment surrounding these systems precipitate those instances remains murky. This paper draws from anthropological work on bureaucracies, states, and power, translating these ideas into a theory describing the structural tendency for powerful algorithmic systems to cause tremendous harm. I show how administrative models and projections of the world create marginalization, just as algorithmic models cause representational and allocative harm. This paper concludes with a recommendation to avoid the absurdity algorithmic systems produce by denying them power.

Integrity signals

Registry

Crossref · 10.1145/3411764.3445740

Cites

64 of 64 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Algorithmic bias can occur in telematics-based scoring, penalizing users whose driving behavior does not correspond to algorithmic imaginations.

    “However, an algorithmic bias can also occur here, which penalizes users with a driving behaviors that do not correspond to the 'algorithmic imaginations'.”

    From Trust-Building in Peer-to-Peer Carsharing: Design Case Study for Algorithm-Based Reputation Systems (2024) · cites this paper as Alkhatib, 2021

  • Regarding trust in the Driving Score, individualizability, e.g., in the form of individual norms, can help to make the system more flexible in order to reduce algorithmic bias as well.

    “Regarding trust in the Driving Score, individualizability, e.g., in the form of individual norms, can help to make the system more flexible (Alkhatib, 2021) in order to reduce algorithmic bias as well.”

    From Trust-Building in Peer-to-Peer Carsharing: Design Case Study for Algorithm-Based Reputation Systems (2024) · cites this paper as Alkhatib, 2021

  • Algorithmic bias can occur in algorithm-based reputation systems, potentially disadvantaging some users or driving behaviors in specific situations.

    “Nonetheless, algorithmic bias can occur here as well, which may disadvantage some users or driving behaviors in specific situations (Jackson Jr, John L, 2013; Alkhatib, 2021).”

    From Trust-Building in Peer-to-Peer Carsharing: Design Case Study for Algorithm-Based Reputation Systems (2024) · cites this paper as Alkhatib, 2021

  • The findings confirm prior work on bias in algorithm-based decisions and the importance of subjective perception of individuals.

    “This confirms the findings of Rouvroy and Alkhatib, whose work draws attention to a bias in algorithm-based decisions and seeks to focus on the importance of subjective perception of individuals (Rouvroy, 2013; Alkhatib, 2021).”

    From Trust-Building in Peer-to-Peer Carsharing: Design Case Study for Algorithm-Based Reputation Systems (2024) · cites this paper as Alkhatib, 2021

  • Algorithmic systems have a structural tendency to specifically misread marginalized groups.

    “This could lead to a structural tendency of algorithmic systems to specifically misread marginalized groups (Alkhatib, 2021).”

    From Trust-Building in Peer-to-Peer Carsharing: Design Case Study for Algorithm-Based Reputation Systems (2024) · cites this paper as Alkhatib, 2021

Cites

Flagged first, then by citations

1 of the 65 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Alkhatib, A.
Affiliations
University of San Francisco (US)
Corresponding author
Alkhatib, Ali
ORCID
Alkhatib, Ali
Venue
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2021-05-06
Pages
1-9
DOI
10.1145/3411764.3445740
OpenAlex
W3160935449
Cited by
81
References
65
Citations by year
2026: 6 · 2025: 9 · 2024: 24 · 2023: 17 · 2022: 17 · 2021: 8
FWCI
14.92
Type
conference paper
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
Keywords
algorithmic harms, algorithmic decision-making, bureaucratic model, marginalization, representational harm, large language models, anthropomorphism of AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review