CoChat
Sign Up
Cited by 85Game Theory and Voting SystemsEthics and Social Impacts of AIExperimental Behavioral Economics Studies

Human-centred mechanism design with Democratic AI

Open access
Koster, Raphael, Balaguer, Jan, Tacchetti, Andrea, Weinstein, Ari, Zhu, Tina, Hauser, Oliver, Williams, Duncan, Campbell-Gillingham, Lucy +3 more·2022·Nature Human Behaviour
DOI 10.1038/s41562-022-01383-xPMC PMC9584820
Open this paper in CoChat — free accountOpen PDF

Abstract

Building artificial intelligence (AI) that aligns with human values is an unsolved problem. Here we developed a human-in-the-loop research pipeline called Democratic AI, in which reinforcement learning is used to design a social mechanism that humans prefer by majority. A large group of humans played an online investment game that involved deciding whether to keep a monetary endowment or to share it with others for collective benefit. Shared revenue was returned to players under two different redistribution mechanisms, one designed by the AI and the other by humans. The AI discovered a mechanism that redressed initial wealth imbalance, sanctioned free riders and successfully won the majority vote. By optimizing for human preferences, Democratic AI offers a proof of concept for value-aligned policy innovation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41562-022-01383-x

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In applying tools from AI research to study resource allocation, we are building upon earlier work, in which we used deep RL to maximise reported human preferences (votes) over a resource allocation policy in a linear public goods game.

    “In applying tools from AI research to study resource allocation, we are building upon earlier work, in which we used deep RL to maximise reported human preferences (votes) over a resource allocation policy in a linear public goods game24,25.”

    From Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem (2025) · cites this paper as [24]

  • In a different game without a common pool, a mixture of progressive redistribution and sanction is the recipe that encourages participants to vote for the mechanism.

    “In a different game without a common pool, this mixture of progressive redistribution and sanction is also the recipe that encourages participants to vote for the mechanism24.”

    From Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem (2025) · cites this paper as [24]

  • A number of interesting ideas are emerging, from voting on different algorithmic policymakers to using LLMs to summarize diverse human opinions and generate consensus statements.

    “A number of interesting ideas are emerging, from voting on different algorithmic policymakers 184 to using LLMs to summarize diverse human opinions 185 and generate consensus statements 153 .”

    From Machine culture (2023) · cites this paper as [185]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. [10]Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at https://arxiv.org/abs/1702.08608 (2017). · arXiv
  2. [11]Lee MK WeBuildAI: participatory framework for algorithmic governance Proc. ACM Hum. Comput. Interact. 2019 3 1 35
  3. [16]Olson, M. The Logic of Collective Action: Public Goods and the Theory of Groups (Harvard Univ. Press, 2003).
  4. [18]Rawls, J. A Theory of Justice (Oxford Univ. Press, 2000).
  5. [30]Schulman J Heess N Weber T Abbeel P Gradient estimation using stochastic computation graphs NeurIPS 2015 28 3528 3536
  6. [34]Strouse DJ McKee K Botvinick MM Hughes E Everett R Collaborating with Humans without Human Data NeurIPS 2021 34 14502 14515
  7. [46]Peters, B. G. The Politics of Bureaucracy: an Introduction to Comparative Public Administration (Routledge, 2010).

Cited by

Sorted by citations

Related

Authors
Koster, R., Balaguer, J., Tacchetti, A., Weinstein, A., Zhu, T., Hauser, O., Williams, D., Campbell-Gillingham, L., Thacker, P., Botvinick, M., & Summerfield, C.
Affiliations
  • Google DeepMind (United Kingdom) (GB)
  • University of Exeter (GB)
  • University of Oxford (GB)
Corresponding author
Summerfield, Christopher
ORCID
Venue
Nature Human Behaviour
Publisher
Springer Science and Business Media LLC
Published
2022-07-04
Volume
6
Issue
10
Pages
1398-1407
ISSN
2397-3374 (ISSN-L)
DOI
10.1038/s41562-022-01383-x
PMC
PMC9584820
OpenAlex
W4283789234
Cited by
84
References
35
Citations by year
2026: 15 · 2025: 22 · 2024: 22 · 2023: 20 · 2022: 5
FWCI
13.86
Type
journal article
Language
en
Update notices
correction (2022-08-12)
Open access
Licence
Locations
  • Nature Human Behaviour (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • Open Research Exeter (University of Exeter) (submittedVersion, cc-by)
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Safety Research › Experimental Behavioral Economics Studies
Keywords
value alignment, human-in-the-loop, reinforcement learning, social mechanisms, majority voting, redistribution mechanisms, wealth inequality, free-riding, collective benefit
MeSH terms
Artificial Intelligence, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review