Using the Veil of Ignorance to align AI systems with principles of justice
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
The philosopher John Rawls proposed the Veil of Ignorance (VoI) as a thought experiment to identify fair principles for governing a society. Here, we apply the VoI to an important governance domain: artificial intelligence (AI). In five incentive-compatible studies ( N = 2, 508), including two preregistered protocols, participants choose principles to govern an Artificial Intelligence (AI) assistant from behind the veil: that is, without knowledge of their own relative position in the group. Compared to participants who have this information, we find a consistent preference for a principle that instructs the AI assistant to prioritize the worst-off. Neither risk attitudes nor political preferences adequately explain these choices. Instead, they appear to be driven by elevated concerns about fairness: Without prompting, participants who reason behind the VoI more frequently explain their choice in terms of fairness, compared to those in the Control condition. Moreover, we find initial support for the ability of the VoI to elicit more robust preferences: In the studies presented here, the VoI increases the likelihood of participants continuing to endorse their initial choice in a subsequent round where they know how they will be affected by the AI intervention and have a self-interested motivation to change their mind. These results emerge in both a descriptive and an immersive game. Our findings suggest that the VoI may be a suitable mechanism for selecting distributive principles to govern AI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Searle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Justice as Fairness: A Restatement
Rawls, J. (2001) · doi:10.2307/j.ctv31xf5v0 · Cited by 5,761
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774
Liberals and conservatives rely on different sets of moral foundations.
Graham, J., Haidt, J., & Nosek, B. A. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015141 · Cited by 4,560
Ericsson (1993) · doi:10.1037/0033-295x.87.3.215 · Cited by 4,539
What Should Be Done with Equity Theory?
Leventhal, G. S. (1980) · Social Exchange · doi:10.1007/978-1-4613-3087-5_2 · Cited by 3,010
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
Principles alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859
Data quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Building a Better America—One Wealth Quintile at a Time
Norton, M. I., & Ariely, D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393524 · Cited by 1,119
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Mohamed, S., Png, M.-T., & Isaac, W. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00405-8 · Cited by 912
Artificial Intelligence, Values, and Alignment
Gabriel, I. (2020) · Minds and Machines · doi:10.1007/s11023-020-09539-2 · Cited by 896
Self-citationToward understanding the impact of artificial intelligence on labor
Frank, M. R., Autor, D., Bessen, J. E., Brynjolfsson, E., Cebrian, M., Deming, D. J., Feldman, M., Groh, M., Lobo, J., Moro, E., et al. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900949116 · Cited by 713
Ross, D. (2002) · doi:10.1093/0199252653.001.0001 · Cited by 658
Recommender systems and their ethical challenges
Milano, S., Taddeo, M., & Floridi, L. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00950-y · Cited by 556
Society-in-the-loop: programming the algorithmic social contract
Rahwan, I. (2018) · Ethics and Information Technology · doi:10.1007/s10676-017-9430-8 · Cited by 553
Algorithmic Accountability and Public Reason
Binns, R. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0263-5 · Cited by 457
Mapping the Landscape of Human-Level Artificial General Intelligence
Adams, S. S., Arel, I., Bach, J., Coop, R., Furlan, R., Goertzel, B., Hall, J. S., Samsonovich, A., Scheutz, M., Schlesinger, M., et al. (2012) · AI Magazine · doi:10.1609/aimag.v33i1.2322 · Cited by 213
On the Utility of Learning about Humans for Human-AI Coordination
Carroll, M., Shah, R., Ho, M. K., Griffiths, T. L., Seshia, S. A., Abbeel, P., & Dragan, A. (2019) · arXiv · doi:10.48550/arxiv.1910.05789 · Cited by 168
Measuring algorithmically infused societies
Wagner, C., Strohmaier, M., Olteanu, A., Kıcıman, E., Contractor, N., & Eliassi-Rad, T. (2021) · Nature · doi:10.1038/s41586-021-03666-1 · Cited by 144
Hard choices in artificial intelligence
Dobbe, R., Krendl Gilbert, T., & Mintz, Y. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103555 · Cited by 65
Cited by
Can Generative AI Improve Social Science?
Bail, C. A. (2023) · doi:10.31235/osf.io/rwtzs · Cited by 369
Can Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369
STELA: a community-centred approach to norm elicitation for AI alignment
Bergman, S., Marchal, N., Mellor, J., Mohamed, S., Gabriel, I., & Isaac, W. (2024) · Scientific Reports · doi:10.1038/s41598-024-56648-4 · Cited by 48
“Among recent efforts, Weidinger et al.’s 59 use of the Veil of Ignorance experiment to select principles to guide AI behaviour.”
BackgroundSelf-citationScaffolding cooperation in human groups with deep reinforcement learning
McKee, K. R., Tacchetti, A., Bakker, M. A., Balaguer, J., Campbell-Gillingham, L., Everett, R., & Botvinick, M. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01686-7 · Cited by 43
Self-citationAI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War
Humphreys, D. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00928-y · Cited by 26
“A further suggestion to minimising bias is through using justice theories such as Rawls’ Veil of Ignorance to align AI sys - tems (Weidinger et al., 2023 ).”
Metadata
- Authors
- Weidinger, L., McKee, K. R., Everett, R., Huang, S., Zhu, T. O., Chadwick, M. J., Summerfield, C., & Gabriel, I.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2023-04-24
- Volume
- 120
- Issue
- 18
- Pages
- e2213709120-e2213709120
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W4366817401
- Cited by
- 43
- References
- 23
- FWCI
- 10.50
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.