CoChat
Sign Up
PreprintCited by 19Evolutionary Game Theory and CooperationExperimental Behavioral Economics StudiesGame Theory and Applications

Learning Reciprocity in Complex Sequential Social Dilemmas

Open access
Eccles, Tom, Hughes, Edward, Kramár, János, Wheelwright, Steven, Leibo, Joel Z.·2019·arXiv
DOI 10.48550/arxiv.1903.08082arXiv 1903.08082
Open this paper in CoChat — free accountOpen PDF

Abstract

Reciprocity is an important feature of human social interaction and underpins our cooperative nature. What is more, simple forms of reciprocity have proved remarkably resilient in matrix game social dilemmas. Most famously, the tit-for-tat strategy performs very well in tournaments of Prisoner's Dilemma. Unfortunately this strategy is not readily applicable to the real world, in which options to cooperate or defect are temporally and spatially extended. Here, we present a general online reinforcement learning algorithm that displays reciprocal behavior towards its co-players. We show that it can induce pro-social outcomes for the wider group when learning alongside selfish agents, both in a $2$-player Markov game, and in $5$-player intertemporal social dilemmas. We analyse the resulting policies to show that the reciprocating agents are strongly influenced by their co-players' behavior.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1903.08082

Cites

49 of 50 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Reputation alone is not a strong factor in the emergence of cooperation within populations; instead, it acts in collaboration with other mechanisms such as punishment.

    “Therefore, reputation alone is not a strong factor in the emergence of cooperation within populations. Instead, it acts in collaboration with other mechanisms such as punishment [13, 15, 20, 21, 60].”

    From Investigating the impact of direct punishment on the emergence of cooperation in multi-agent reinforcement learning systems (2025) · cites this paper as [60]

    The Introduction (p. 1) says reciprocity can foster collective action by rewarding pro-social behavior and punishing anti-social acts, but it does not assess reputation. Searches for “reputation” and for reputation acting with punishment found no passage addressing whether reputation alone is weak or works alongside punishment. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. [6][6] L.C. Becker. 1990. Reciprocity . University of Chicago Press. https://books.google. co.uk/books?id=dWgI4lI7h-cC -
  2. [20][20] Joanna Hale and Antonia F. de C. Hamilton. 2016. Cognitive mechanisms for responding to mimicry from others. Neuroscience & Biobehavioral Reviews 63 [(2016), 106 - 123. https://doi.org/10.1016/j.neubiorev.2016.02.006](https://doi.org/10.1016/j.neubiorev.2016.02.006) - · DOI
  3. [34][34] A. Lerer and A. Peysakhovich. 2018. Learning Existing Social Conventions in Markov Games. ArXiv e-prints (June 2018). arXiv:cs.AI/1806.10071 -
  4. [40][40] M. A. Nowak. 2006. Evolutionary Dynamics . Harvard University Press. https: //books.google.co.uk/books?id=YXrIRDuAbE0C -
  5. [55][55] J.W.A. THIBAUT and H.H. Kelley. 1966. The Social Psychology of Groups . Wiley. https://books.google.co.uk/books?id=KDH5Hc9F2AkC -

Cited by

Sorted by citations

Related

Authors
Eccles, T., Hughes, E., Kramár, J., Wheelwright, S., & Leibo, J. Z.
Affiliations
Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1903.08082
arXiv
1903.08082
OpenAlex
W2921919225
Cited by
19
References
50
Citations by year
2024: 2 · 2023: 3 · 2022: 2 · 2021: 7 · 2020: 5
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Sociology and Political Science › Evolutionary Game Theory and Cooperation
  • Social Sciences › Safety Research › Experimental Behavioral Economics Studies
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
Keywords
reciprocity, social dilemmas, online reinforcement learning, stochastic games, cooperation, tit-for-tat, prosocial behavior, multi-agent learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review