Cited by 76Experimental Behavioral Economics StudiesSocial Robot Interaction and HRIEvolutionary Game Theory and Cooperation

Algorithm exploitation: Humans are keen to exploit benevolent AI

Open access
Karpus, Jurgis, Krüger, Adrian, Verba, Julia Tovar, Bahrami, Bahador, Deroy, Ophelia·2021·iScience

Abstract

We cooperate with other people despite the risk of being exploited or hurt. If future artificial intelligence (AI) systems are benevolent and cooperative toward us, what will we do in return? Here we show that our cooperative dispositions are weaker when we interact with AI. In nine experiments, humans interacted with either another human or an AI agent in four classic social dilemma economic games and a newly designed game of Reciprocity that we introduce here. Contrary to the hypothesis that people mistrust algorithms, participants trusted their AI partners to be as cooperative as humans. However, they did not return AI's benevolence as much and exploited the AI more than humans. These findings warn that future self-driving cars or co-working robots, whose success depends on humans' returning their cooperativeness, run the risk of being exploited. This vulnerability calls not just for smarter machines but also better human-centered policies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.isci.2021.102679

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Karpus et al. [61] found that while users of automated vehicles are well aware of traffic rules and the role of automated vehicles on the road, drivers are keen on exploiting automated cars as compared to cars driven by human drivers.

    “Karpus et al. [61] have found that while users of automated vehicles are well aware of traffic rules and the role of automated vehicles on the road, drivers are keen on exploiting automated cars as compared to cars driven by human drivers.”

    From The AI Ghostwriter Effect: When Users do not Perceive Ownership of AI-Generated Text but Self-Declare as Authors (2024) · cites this paper as Karpus et al. [61]

    The Introduction proposes that people may exploit a cooperative AI agent more than a human, and the Discussion treats greater exploitation of cooperative artificial agents as a finding. But the paper uses driving as an illustration, not as the setting of its experiments; it does not establish that drivers exploit automated cars on the road or that automated-vehicle users are aware of traffic rules and the vehicles’ role. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Studies have shown that people are more inclined to exploit algorithms or AI models than humans.

    “Studies have shown that people are more inclined to exploit algorithms or AI models than humans [61].”
  • The human ghostwriter was credited more often than the AI ghostwriter (H5).

    “In line with the tendency of humans to more readily exploit AI than a human [61], the human ghostwriter was also credited more often than the AI ghostwriter (H5).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

4 of the 56 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Colman A.M. Game Theory & its Applications in the Social and Biological Sciences 1999 Routledge
  2. Rahwan I. Cebrian M. Obradovich N. Bongard J. Bonnefon J.-F. Breazeal C. Crandall J.W. Christakis N.A. Couzin I.D. Jackson M.O. Machine behavior Nature 568 2019 477 486 31019318

Cited by

Sorted by citations

Authors
Karpus, J., Krüger, A., Verba, J. T., Bahrami, B., & Deroy, O.
Affiliations
  • Ludwig-Maximilians-Universität München (DE)
  • University of London (GB)
Corresponding author
Deroy, Ophelia
Venue
iScience
Publisher
Elsevier BV
Published
2021-06
Volume
24
Issue
6
Pages
102679
ISSN
2589-0042 (ISSN-L)
OpenAlex
W3170088386
Cited by
76
References
56
Citations by year
2026: 20 · 2025: 27 · 2024: 13 · 2023: 10 · 2022: 4 · 2021: 2
FWCI
9.07
Type
journal article
Language
en
Open access
Locations
  • iScience (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PhilPapers (PhilPapers Foundation) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, cc-by-nc-nd)
Funders
  • Ludwig-Maximilians-Universität München
  • European Research Council
  • Horizon 2020 (819040)
  • Bundesministerium für Bildung und Forschung
  • NOMIS Stiftung
  • Alexander von Humboldt-Stiftung
  • Norges Forskningsråd
Topics
  • Social Sciences › Safety Research › Experimental Behavioral Economics Studies
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Social Sciences › Sociology and Political Science › Evolutionary Game Theory and Cooperation
Keywords
human-AI interaction, reciprocity, social dilemmas, cooperation, exploitation, trust in algorithms, economic games, autonomous vehicles

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review