The DOI matches its Crossref record; 278 of the 283 references it cites were found in a registry or an index.
Responses to catastrophic AGI risk: a survey
Open accessAbstract
Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years.It has been suggested that AGI may inflict serious damage to human well-being on a global scale ('catastrophic risk').After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk.We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that are safe due to their internal design.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497
Seeing like a State: How Certain Schemes to Improve the Human Condition Have Failed
Fukuyama, F., & Scott, J. C. (1998) · Foreign Affairs · doi:10.2307/20048980 · Cited by 8,396
Construal-level theory of psychological distance.
Trope, Y., & Liberman, N. (2010) · Psychological Review · doi:10.1037/a0018963 · Cited by 6,618
Freedom of the Will and the Concept of a Person
Frankfurt, H. G. (1971) · The Journal of Philosophy · doi:10.2307/2024717 · Cited by 4,663
Validity of the single processor approach to achieving large scale computing capabilities
Amdahl, G. M. (1967) · Proceedings of the April 18-20, 1967, spring joint computer conference on - AFIPS '67 (Spring) · doi:10.1145/1465482.1465560 · Cited by 4,339
Seeing like a state: how certain schemes to improve the human condition have failed
(1998) · Choice Reviews Online · doi:10.5860/choice.36-1224 · Cited by 4,081
Searle, J. R. (1992) · doi:10.7551/mitpress/5834.001.0001 · Cited by 3,595
The Rise and Decline of Nations.
Beckerman, W., & Olson, M. (1983) · The Economic Journal · doi:10.2307/2232759 · Cited by 2,792
The blank slate: the modern denial of human nature
(2003) · Choice Reviews Online · doi:10.5860/choice.40-4305 · Cited by 2,443
Robust De-anonymization of Large Sparse Datasets
Narayanan, A., & Shmatikov, V. (2008) · 2008 IEEE Symposium on Security and Privacy (sp 2008) · doi:10.1109/sp.2008.33 · Cited by 2,372
Research works printed but not linked: 84
- Agar N, 2011, J. Evolution Technol, 22, 23
- Amdahl G M, 1967, Proc. Spring Joint Computer Conference (AFIPS ‘67), 483
- Armstrong S, 2012, Proc. Int. Conf. Beyond AI 2012, 5–6 November 2012, Pilsen, Czech Republic
- Asaro P M, Proc. IEEE Conf. on Robotics and Automation, Workshop on Roboethics
- Baars B J ed Laureys S, 2005, The Boundaries of Consciousness, 45
- Bach J, 2012, Artificial General Intelligence (Lecture Notes in Artificial Intelligence
- Beavers A F, 2009, 18th Ann. Meeting of Association for Practical and Professional Ethics, Cincinnati, OH
- Beavers A F, 2012, Robot Ethics: The Ethical and Social Implications of Robotics, 333
- Blackmore S, 2012, J. Consciousness Studies, 19, 16
- Bostrom N, 1998, Int. J. Futures Studies, 2
- Bostrom N, 2002, J. Evolution Technol., 9
- Bostrom N ed Smit I, 2003, Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, 2, 12
- Bryson J, 1998, Just another artefact
- Bugaj S V, 2007, Dynamical Psychology
- Butler S, 1863, The Press
- Calandrino J A, 2011, Proc. 20th USENIX Security Symp, 267
- Cassimatis N, 2006, AI Mag., 27, 12
- Casti J L, 2012, X-Events
- Chalmers D J, 1996, The Conscious Mind (Philosophy of Mind Series)
- Chalmers D J, 2010, J. Consciousness Studies, 17, 7
- Paul F, 2012, Ordinary Ideas
- Daley W, 2011, Synthese, 2, 44
- de Garis H, 2005, The Artilect War: Cosmists vs Terrans
- Dennett D C ed Pylyshyn Z W, 1987, The Robot’s Dilemma, 41
- Dennett D C, 2012, J. Consciousness Studies, 19, 86
- Dietrich E, 2014, Philosophy Now
- Fox J, 2010, ECAP10, VIII European Conference of Computing and Philosophy
- Franklin S, 2006, IDPT-2006 Proc.
- Goertzel B, 2010, The Multiverse According to Ben
- Goertzel B, 2002, Dynamical Psychology
- Goertzel B, 2004, Dynamical Psychology
- Goertzel B, 2004, Dynamical Psychology
- Goertzel B, 2012, J. Consciousness Studies, 19, 96
- Goertzel B, 2012, J. Evolution Technol., 22, 116
- Gunkel D J, 2012, The Machine Question, 10.7551/mitpress/8975.001.0001 · DOI · Paper page
- Guterl F, 2012, The Fate of the Species
- Hall J S, 2007, Beyond AI
- Hall J S ed Wang P, 2008, Artificial General Intelligence Frontiers, 460
- Hall J S ed Anderson M, Machine Ethics, 512
- Hanson R, 2009, Overcoming Bias
- Hanson R, 1994, Extropy, 6
- Hanson R, 2012, J. Consciousness Studies, 19, 119
- Heylighen F, 2012, J. Consciousness Studies, 19, 126
- Hibbard B ed Wang P, 2008, Artificial General Intelligence Frontiers, 473
- Hutter M, 2012, J. Consciousness Studies, 19, 143
- Karnofsky H, 2012, Less Wrong
- Legg S, 2009, Funding safe AGI Vetta Project
- Lewis D, 1989, Proc. Aristotelian Soc. Suppl. Volumes, 63, 113
- Mainzer K (ed), 2010, ECAP10, VIII European Conference of Computing and Philosophy
- McDermott D, 2012, J. Consciousness Studies, 19, 167
- McKibben B, 2003, Enough
- Miller J D, 2012, Singularity Rising
- Minsky M, 2004, AI Mag., 25, 113
- Moravec H P, 1988, Mind Children
- Moravec H P, 1998, J. Evolution Technol., 1
- Moravec H P, 1999, Robot
- Muehlhauser L, 2012, Less Wrong
- Napier W ed Bostrom N, 2011, Global Catastrophic Risks, 222
- Narayanan A, 2009, 30th IEEE Symp. on Security and Privacy, 173
- Nordmann A, 2008, IEEE Spectrum
- Omohundro S M ed Wang P, 2008, Artificial General Intelligence Frontiers, 483
- Rajab M A, 2007, Proc. of 1st Workshop on Hot Topics in Understanding Botnets (HotBots '07)
- Ramamurthy U, 2006, Proc. Seventh International Conference on Cognitive Modeling, 244
- Salekin R T ed Salekin R T, 2010, Treatment of child and adolescent psychopathy Handbook of Child and Adolescent Psychopathy, 343
- Sandberg A ed Eden A H, 2012, Singularity Hypotheses
- Scott J C, 1998, Seeing Like a State
- Shulman C, 2010, ECAP10, VIII European Conference of Computing and Philosophy
- Shulman C, 2009, AP-CAP 2009, 23
- Shulman C, 2009, AP-CAP 2009, 95
- Solum L B, 1992, North Carolina Law Rev., 70, 1231
- Stahl B C, 2002, 14th Int. Conf. on Systems Research, Informatics and Cybernetics: Symposium on Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, 13
- Staniford S, 2002, Proc. 11th USENIX Security Symp., 149
- Sullins J P, 2006, Int. Rev. Inf. Ethics, 6, 23
- Turney P, 1991, Can. Artificial Intell., 27, 3
- Verdoux P, 2010, J. Futures Studies, 15, 1
- Walker M, 2011, J. Evolution Technol., 22, 37
- Wang P, 2008, Artificial General Intelligence Frontiers
- Warwick K, 1998, In the Mind of the Machine
- Weld D, 1994, Proc. Twelfth National Conf. on Artificial Intelligence, 1042
- Wilson G S, 2014, Law J., 31, 307
- Yampolskiy R V, 2012, J. Consciousness Studies, 1, 194
- Yudkowsky E, 2008, Less Wrong
- Yudkowsky E, 2009, Less Wrong
- Yudkowsky E, 2012, Less Wrong
Cited by
The ai-based cyber threat landscape: A survey
Kaloudi, N., & Li, J. (2021) · ACM Computing Surveys · doi:10.1145/3372823 · Cited by 310
“The authors [76] mainly emphasized the domain of AI safety engineering and its associated needs to create a safe machine.”
BackgroundInfluentialThe risks associated with Artificial General Intelligence: A systematic review
McLean, S., Read, G. J. M., Thompson, J., Baber, C., Stanton, N. A., & Salmon, P. M. (2023) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2021.1964003 · Cited by 203
Everitt, T., Lea, G., & Hütter, M. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/768 · Cited by 197
On the promotion of safe and socially beneficial artificial intelligence
Baum, S. D. (2017) · AI & SOCIETY · doi:10.1007/s00146-016-0677-0 · Cited by 113
Classification of global catastrophic risks connected with artificial intelligence
Turchin, A., & Denkenberger, D. (2018) · AI & Society · doi:10.1007/s00146-018-0845-5 · Cited by 199
Metadata
- Authors
- Sotala, K., & Yampolskiy, R. V.
- Affiliations
- Machine Intelligence Research Institute (US)
- University of Louisville (US)
- Venue
- Physica Scripta
- Publisher
- IOP Publishing
- Published
- 2014-12-19
- Volume
- 90
- Issue
- 1
- Pages
- 018001
- ISSN
- 0031-8949 (ISSN-L)
- 1402-4896
- OpenAlex
- W2157925625
- Cited by
- 121
- References
- 283
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 7 · 2023: 8 · 2022: 10 · 2021: 9 · 2020: 15 · 2019: 6 · 2018: 12 · 2017: 12 · 2016: 12 · 2015: 10 · 2014: 4
- FWCI
- 35.27
- Type
- journal article
- Language
- en
- Update notices
- correction (2017-01-26)
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- iopscience.iop.org/info/page/text-and-data-mining (tdm)
- creativecommons.org/licenses/by/3.0/ (vor)
- Locations
- Physica Scripta (publishedVersion, cc-by)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- ThinkIR: The University of Louisville's Institutional Repository (University of Louisville) (submittedVersion)
- Funders
- Machine Intelligence Research Institute
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- artificial general intelligence, catastrophic risk, AI safety, AI alignment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Responses to catastrophic AGI risk: a survey. https://app.cochat.ai/papers/10.1088/0031-8949/90/1/018001
- The paper
- Sotala, K., & Yampolskiy, R. V. (2015). Responses to catastrophic AGI risk: a survey. Physica Scripta. https://doi.org/10.1088/0031-8949/90/1/018001
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.