Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 99 of the 101 references it cites were found in a registry or an index; 5 of the 66 sources its claims cite were checked for support.

Artificial Intelligence, Values, and Alignment

Open access
Gabriel, Iason · 2020 · Minds and Machines

Abstract

This paper looks at philosophical questions that arise in the context of AI alignment. It defends three propositions. First, normative and technical aspects of the AI alignment problem are interrelated, creating space for productive engagement between people working in both domains. Second, it is important to be clear about the goal of alignment. There are significant differences between AI that aligns with instructions, intentions, revealed preferences, ideal preferences, interests and values. A principle-based approach to AI alignment, which combines these elements in a systematic way, has considerable advantages in this context. Third, the central challenge for theorists is not to identify ‘true’ moral principles for AI; rather, it is to identify fair principles for alignment that receive reflective endorsement despite widespread variation in people’s moral beliefs. The final part of the paper explores three ways in which fair principles for AI alignment could potentially be identified.

Overview

CoChat Summary

## TL;DR

This article argues that AI alignment is both a technical and a normative problem, so the choice of values cannot be separated from how systems are built. It also argues that alignment should target fair, publicly justifiable principles rather than a single “true” moral theory, because people hold diverse reasonable beliefs about value.

## Key findings

  • The paper says the technical and normative parts of alignment are “not orthogonal but rather related in important ways,” and that “the way we build AI is likely to influence the values we are able to load.”
  • It distinguishes several possible alignment targets—instructions, intentions, revealed preferences, informed preferences, interests/well-being, and values—and argues that none of the earlier targets is sufficient on its own.
  • On revealed preferences, the paper notes a practical problem of “degeneracy”: at any moment there is “a large set of reward functions for which an observed behaviour is optimal,” making reliable inference hard; it also says revealed preferences can misfire in self-harm, harm to others, and adapted preferences.
  • It reports that Jobin et al. (2019) analyzed 84 AI ethics documents and found a “global convergence” around five principles: transparency, justice and fairness, non-maleficence, responsibility, and privacy.

## Methods

  • Conceptual/philosophical argument, not an empirical study.
  • The paper reviews AI alignment concepts from machine learning and reinforcement learning, then maps them onto normative options from moral and political philosophy.
  • It develops three candidate routes for selecting fair alignment principles: overlapping consensus / global public morality and human rights, hypothetical agreement behind a veil of ignorance, and social choice / democratic aggregation.

## Limitations

  • The paper states that there is “no way to ‘bracket out’ normative questions altogether,” so technical methods alone cannot solve alignment.
  • It notes that existing AI ethics principles are often too abstract and vague to be action-guiding, and that apparent consensus may hide deep disagreement; this is a limitation of the field the paper discusses, not a study limitation.
  • Evident from the text: the article is largely theoretical and does not present new data, experiments, or a formal proof of the proposed principles.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11023-020-09539-2

Cites

99 of 101 references verified1 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

5 Valid · 0 Not valid · 49 Not checked · 82 Own results and interpretations

61 more claims

Create a free account to see all 136 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 57
  • It may state two separate claims as one. 23
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 91 references

Research works printed but not linked: 23

  1. Asilomar, A.I. (2018, October). Principles (2017). In Principles developed in conjunction with the 2017 Asilomar conference [Benevolent AI 2017].
  2. Asimov, I. (2004). I, robot (Vol. 1). Spectra.
  3. Blackburn, S. (2001). Ruling passions: An essay in practical reasoning. Oxford: Oxford University Press.
  4. Bostrom, N. (2016). Superintelligence: Paths, dangers, strategies. Oxford: Oxford University Press.
  5. Bratman, M. (1987). Intention, plans, and practical reason (Vol. 10). Cambridge, MA: Harvard University Press.
  6. Dennett, D. (2003). Freedom Evolves. New York: Penguin Books.
  7. Dworkin, R. (1981). What is equality? Part 1: Equality of welfare. in Philosophy & public affairs, (pp. 185–246).
  8. Dworkin, R. (1984). Rights as Trumps. Arguing Law (pp. 335–44).
  9. Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., et al. (2018). De novo structure prediction with deeplearning based scoring. Annual Review of Biochemistry, 77(363–382), 6.
  10. MacIntyre, A. (2013). After virtue. London: A&C Black.
  11. Mill, J.S. (1859/1998). On liberty and other essays. Oxford: Oxford University Press.
  12. Nagel, T. (1989). The view from nowhere. Oxford: Oxford University Press.
  13. Nozick, R. (1974). Anarchy, state, and utopia. New York: Basic Books.
  14. Parfit, D. (2011). On what matters (Vol. 1). Oxford: Oxford University Press.
  15. Rorty, R. (1993). Human rights, rationality, and sentimentality. Wronging Rights Philos. Chall. Hum. Rights (pp. 1–34).
  16. Russell, S. (2019). Human Compatible: AI and the Problem of Control. Bristol: Allen Lane.
  17. Russell, S., & Norvig, P. (2010). Artificial intelligence: A modern approach (3rd ed.). Upper Saddle River: Prentice Hall.
  18. Shue, H. (1996). Basic rights: Subsistence, affluence, and US foreign policy. Princeton: Princeton University Press.
  19. Smith, M. R. (1994). The Moral Problem. New York: Malden Blackwell.
  20. Soares, N. (2014). The Value Learning Problem, Technical Report. Berkeley, CA: Machine Intelligence Research Institute.
  21. Taylor, J. (2016). Quantilizers: A Safer Alternative to Maximizers for Limited Optimization, in Presented at the Workshops at the Thirtieth AAAI Conference on Artificial Intelligence.
  22. UN Secretary General. (2019). The age of digital interdependence, Report of the UN Secretary General’s High-Level Panel on Digital Cooperation. Geneva: Un Secretary General.
  23. Vaillant, G. E. (2008). Aging well: Surprising guideposts to a happier life from the landmark study of adult development. Brown: Little.

Cited by

See all 896

Metadata

Authors
Gabriel, I.
Affiliations
Google DeepMind (United Kingdom) (GB)
Corresponding author
Gabriel, Iason
Venue
Minds and Machines
Published
2020-09
Volume
30
Issue
3
Pages
411-437
ISSN
  • 0924-6495 (ISSN-L)
  • 1572-8641
OpenAlex
W3002093512
Cited by
896
References
91
Citations by year
2026: 274 · 2025: 229 · 2024: 164 · 2023: 93 · 2022: 64 · 2021: 61 · 2020: 11
FWCI
77.92
Type
journal article
Language
en
Open access
Licence
Locations
  • Minds and Machines (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Arts and Humanities › Philosophy › Philosophical Ethics and Theory
  • Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
Keywords
AI alignment, AI ethics, moral values, revealed preference, interest, FAIR principles, moral pluralism

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Artificial Intelligence, Values, and Alignment. https://app.cochat.ai/papers/10.1007/s11023-020-09539-2
The paper
Gabriel, I. (2020). Artificial Intelligence, Values, and Alignment. Minds and Machines. https://doi.org/10.1007/s11023-020-09539-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.