The DOI matches its DataCite record.
Low Impact Artificial Intelligences
Abstract
There are many goals for an AI that could become dangerous if the AI becomes superintelligent or otherwise powerful. Much work on the AI control problem has been focused on constructing AI goals that are safe even for such AIs. This paper looks at an alternative approach: defining a general concept of `low impact'. The aim is to ensure that a powerful AI which implements low impact will not modify the world extensively, even if it is given a simple or dangerous goal. The paper proposes various ways of defining and grounding low impact, and discusses methods for ensuring that the AI can still be allowed to have a (desired) impact despite the restriction. The end of the paper addresses known issues with this approach and avenues for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 9
- Stuart Armstrong. Motivated value selection for artificial agents. presented at the 1st International Workshop on AI and Ethics, 2015.
- Nick Bostrom. Superintelligence: Paths, dangers, strategies. Oxford University Press, 2014.
- Daniel Dewey. Learning what to value. In Artificial General Intelli- gence, pages 309-314. Springer Berlin Heidelberg, 2011.
- P.D. Grünwald and A.P. Dawid. Game theory, maximum entropy, minimum discrepancy, and robust bayesian decision theory. Annals of Statistics, 32:1367-1433, 2004.
- Nelson Goodman. Fact, fiction, and forecast. Harvard University Press, 1983.
- Tilmann Gneiting and Adrian E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-378, 2007.
- S. Omohundro. Basic ai drives. In Proceedings of the First AGI Conference, volume 171, 2008.
- Roman V. Yampolskiy. Leakproofing the singularity: artificial in- telligence confinement problem. Journal of Consciousness Studies, 19:194-214, 2012.
- Eliezer Yudkowsky. Artificial intelligence as a positive and negative factor in global risk. In Nick Bostrom and Milan M. Ćirković, editors, Global catastrophic risks, pages 308-345, New York, 2008. Oxford University Press.
Cited by
Sorted by citations
Penalizing side effects using stepwise relative reachability
Krakovna, V., Orseau, L., Kumar, R., Martic, M., & Legg, S. (2018) · arXiv · doi:10.48550/arxiv.1806.01186 · Cited by 26
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Dalrymple, D. "., Skalse, J., Bengio, Y., Russell, S., Tegmark, M., Seshia, S., Omohundro, S., Szegedy, C., Goldhaber, B., Ammann, N., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.06624 · Cited by 12
Calibrating Conservatism for Scalable Oversight
Overman, W., & Bayati, M. (2026) · arXiv · doi:10.48550/arxiv.2605.28807
Metadata
- Authors
- Armstrong, S., & Levinstein, B.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- arXiv
- 1705.10720
- References
- 0
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
