Managing extreme AI risks amid rapid progress
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Preparation requires technical research and development, as well as adaptive, proactive governance
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Any advancement that increases the autonomous capabilities of AI systems introduces its own set of safety considerations (Bengio et al., 2024), especially for systems that improve in an open-ended way (Ecoffet et al., 2020; Clune, 2019).”
From Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents (2025) · cites this paper as Bengio et al., 2024
In the citing paper
“Advanced AI also introduces phenomena that differ fundamentally from previous generations of AI or other technologies, requiring new approaches to mitigating these risks (Bengio et al., 2024).”
From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Bengio et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationA Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925 · cited 2×
“However, AI is making progress in critical domains such as hacking, social manipulation, and strategic planning 24,31 , and may soon pose unprecedented con-trol challenges.”
BackgroundEmergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Augmenting large language models with chemistry tools
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00832-8 · Cited by 794
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“Especially in the hands of a few powerful actors, AI could cement or exacerbate global inequities, or facilitate automated warfare, customized mass manipulation, and pervasive surveillance 18–23 .”
BackgroundA graph placement methodology for fast chip design
Mirhoseini, A., Goldie, A., Yazgan, M., Jiang, J. W., Songhori, E., Wang, S., Lee, Y.-J., Johnson, E., Pathak, O., Nova, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03544-w · Cited by 618 · cited 1×
“Progress in AI also enables faster AI progress 8 : AI assistants are increasingly used to automate programming 9 , data collection 10,11 , and chip design 12 .”
MethodSuperhuman AI for multiplayer poker
Brown, N., & Sandholm, T. (2019) · Science · doi:10.1126/science.aay2400 · Cited by 592
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.17580 · Cited by 477
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1657 · Cited by 475
Research works printed but not linked: 8
- M. Kinniment et al. arXiv:2312.11671 (18 December 2023). · arXiv
- I. R.McKenzie. arXiv:2306.09479 (15 June 2023). · arXiv
- R.Ngo L.Chan S.Mindermann arXiv:2209.00626 (20 August 2022). · arXiv
- E.Hubinger. arXiv:2401.05566 (10 January 2024). · arXiv
- L.Koessler J.Schuett arXiv:2307.08823 (17 July 2023). · arXiv
- N. Kolt et al. arXiv:2404.02675 (3 April 2024). · arXiv
- M. Phuong et al. arXiv:2403.13793 (20 March 2024). · arXiv
- T. A. Kelly, SAE Trans. J. Mater. Manuf. 113, 257 (2004).
Cited by
Items 301–400 of 618
Zhong, L., & Ma, X. (2026) · British Educational Research Journal · doi:10.1002/berj.70098 · Cited by 2
The Conceptual Architecture of Digital Governance: A Systematic Review (2015-2025)
Alexander, L. E. E. (2025) · Digital Society & Virtual Governance · doi:10.6914/esjj0n53 · Cited by 3
“The first, primarily technical, focuses on alignment and safety — ensuring that autonomous systems pursue intended objectives (Ngo et al., 2022; Bengio et al., 2024).”
Artificial Intelligence at the Intersection of Chemistry and Materials Science
Gregan, T., & Gregáň, J. (2026) · AI · doi:10.3390/ai7030089 · Cited by 3
“While researchers have warned of potential risks associated with AI, consensus on how to effectively address them remains limited [62].”
Datko, S., Szyc, K., Walkowiak, T., & Maciejewski, H. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132191 · Cited by 2
Designing Integrated Policies for the Twin Transition: Challenges and Tradeoffs
Maggetti, M. (2026) · Policy Studies Journal · doi:10.1111/psj.70118 · Cited by 2
Hu, Y., Yan, X., Duan, C., Yang, X., & Yang, J. (2026) · Systems · doi:10.3390/systems14040377 · Cited by 2
Layered Control Architectures for AI Safety: A Cybersecurity-Oriented Systems Framework
Choi, Y. B., Hong, P. C., & Park, Y. S. (2026) · Systems · doi:10.3390/systems14040447 · Cited by 2
The Sociology of AI as an Emerging Field: Mapping Tensions and Boundaries
Liu, C. (2026) · Sociology Compass · doi:10.1111/soc4.70201 · Cited by 2
Origins and Evolution of Imagination, From Australopithecus to Modern‐Day Deep Learning
Sidhu, K., Roy, R., & D’Angiulli, A. (2026) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.70030 · Cited by 2
The right balance: how to fix European Union artificial intelligence regulation
Mariniello, M. (2026) · doi:10.64153/bvaq6479 · Cited by 4
Jiang, L., Chen, J., Lu, L., Peng, X., Liu, L., He, J., & Xu, J. (2026) · JMIR AI · doi:10.2196/86864 · Cited by 2
“Third, standardized evaluation methodologies for cross-modal medical tasks (eg, radiology report generation) remain underdeveloped [12].”
iBRAHIMOĞLU, N., Demirezen, M. U., & Yıldız, F. (2026) · Cluster Computing · doi:10.1007/s10586-026-06537-4 · Cited by 3
Do societal promises influence patent value? An analysis of inventions in artificial intelligence
Pelaez, S., Ribeiro, B., Verma, G., & Philip Shapira, A. (2026) · Research Policy · doi:10.1016/j.respol.2026.105566 · Cited by 3
Aspects of Decision-Making in Human–Machine Teaming
Balthasar, M. (2024) · Springer proceedings in complexity · doi:10.1007/978-3-031-57785-7_43 · Cited by 2
Resilience to the Flowing Unknown: An Open Set Recognition Framework for Data Streams
Barcina-Blanco, M., Lobo, J. L., Bringas, P. G., & Ser, J. D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-74183-8_12 · Cited by 3
Designing for the Future of Information Access with Generative Information Retrieval
Murdock, V., Lee, C., & Hersh, W. (2024) · The information retrieval series · doi:10.1007/978-3-031-73147-1_9 · Cited by 2
Revolutionizing Aerial Operations With AI-Enhanced Drones
Palaniappan, D., Premavathi, T., Jain, R., & Parmar, K. J. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-8497-8.ch010 · Cited by 2
People make suboptimal decisions about existential risks
Elga, A., Zhu, J.-Q., & Griffiths, T. L. (2025) · Cognition · doi:10.1016/j.cognition.2025.106216 · Cited by 2
Veloz, T. (2025) · Frontiers in Cognition · doi:10.3389/fcogn.2025.1618381 · Cited by 2
India’s fragmented landscape of AI regulation and governance and the work and employment scenario
Roy, C., & Sarma, K. D. (2025) · Global Political Economy · doi:10.1332/26352257y2025d000000039 · Cited by 2
Understanding frontier AI capabilities and risks through semi-structured interviewsPreprint
Wasil, A. R., Berglund, L., Reed, T., Plueckebaum, M., & Smith, E. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4881729 · Cited by 2
Floch, N. R., & Floch, J. (2024) · Obesity Surgery · doi:10.1007/s11695-024-07450-8 · Cited by 2
Bova, P., Stefano, A. D., & Han, T. A. (2024) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2412.15433 · Cited by 2
“…a selection of additional risks include multi-agent risks, such as collusion between AI systems, systemic risks such as the shrinking of human-agency, and power-seeking behaviour when combined with long-term planning or strategising (Bengio, Hinton, et al., 2024; Hendrycks et al., 2023).”
Li, Z., Lyu, M., Li, L., & Huang, J. (2025) · Research Square · doi:10.21203/rs.3.rs-6468789/v1 · Cited by 2
The Forthcoming AGI Revolution: Its Impact on Society and FirmsPreprint
Makridakis, S., & Michailides, M. (2025) · Preprints.org · doi:10.20944/preprints202508.0201.v1 · Cited by 2
Carroll, B. J., Zhou, J., Burke, P. F., & Ammon, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.11327 · Cited by 2
Head-Specific Intervention Can Induce Misaligned AI Coordination in Large Language ModelsPreprint
Darm, P., & Riccardi, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.05945 · Cited by 2
“Therefore, aligning models to produce safe, ethical, and harmless outputs has become increasingly important (Bengio et al., 2024).”
Underwriting the Agent Economy: The Blueprint for an AI Insurance StackPreprint
Trout, C., Koyejo, S., Romanosky, S., Ripamonti, G., Thompson, L., Spain, D., Taylor, A., Casey, K., Casper, S., Botvinick, M., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.11999 · Cited by 3
Traccia: An OpenTelemetry-Based Governance Platform for AI SystemsPreprint
Naik, N. K., Saroj, A. K., Poudel, V. P., Samantray, S., & Patel, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.14309 · Cited by 3
“Since various orchestration frameworks produce completely heterogeneous logs, it is difficult to aggregate evidence into a consistent transparency register [58, 59, 47].”
Artificial Intelligence and the Future of Finance: A Framework for Structural Change
Naqvi, M. (2026) · doi:10.56227/26.1.16 · Cited by 2
What if automating AI R&D triggers an intelligence explosion?Preprint
Chan, A., Winter, C. K., Barto, A. G., Pachocki, J., Hinton, G. E., Horvitz, E., Bengio, Y., Song, D., Clark, J., Greaves, H., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.36054 · Cited by 2
“Such a loss of control could potentially lead to a range of catastrophic outcomes, including, at the extreme, the marginalization or extinction of humanity [40, 41].”
Self-citationSafety Research and the Allocation of Frontier ComputePreprint
Gan, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.14305 · Cited by 2
Examining popular arguments against AI existential risk: a philosophical analysis
Swoboda, T., Uuk, R., Lauwaert, L., Rebera, A. P., Oimann, A.-K., Chomanski, B., & Prunkl, C. (2026) · Ethics and Information Technology · doi:10.1007/s10676-025-09881-y · Cited by 1
“Discussions of AI as an extinction risk to humanity generally revolve around two key pathways (Bengio et al., 2024; Bostrom, 2014; Carlsmith, 2024; Hendrycks et al., 2023; Russell, 2019).”
Do as We Do, Not as You Think: the Conformity of Large Language Models
Weng, Z., Chen, G., & Wang, W. (2025) · arXiv · doi:10.48550/arxiv.2501.13381 · Cited by 1
“This is a hard, long-standing problem in the field of LLMs since its inception [58–60], and is attracting increasing attention due to the critical importance of AI safety for human society [61].”
AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies
Zeng, Y., Yang, Y., Zhou, A., Tan, J. Z., Tu, Y., Mai, Y., Klyman, K., Pan, M., Jia, R., Song, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.17436 · Cited by 1
Self-citationA Red Teaming Roadmap Towards System-Level Safety
Wang, Z., Knight, C. Q., Kritz, J., Primack, W. E., & Michael, J. (2025) · arXiv · doi:10.48550/arxiv.2506.05376 · Cited by 1
“The rapid advancement of AI capabilities towards general—and even superhuman— intelligence poses serious risks to both individuals and society, but the total investment in AI safety remains minuscule compared to capability-focused research [7, 21].”
Zhang, X., Zhou, W., Hu, W., Zhou, S., Hu, X., & Yang, L. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.102900 · Cited by 1
Balthasar, M. (2024) · i-com · doi:10.1515/icom-2024-0016 · Cited by 1
AI in education: Towards an autotuned magic flute?
Τσώνη, Ρ., & Verykios, V. S. (2024) · Διεθνές Συνέδριο για την Ανοικτή & εξ Αποστάσεως Εκπαίδευση · doi:10.12681/icodl.6013 · Cited by 2
Dinámicas de la violencia en una realidad disruptiva: la deshumanización acelerada por la IA
Cordova, R. B. (2024) · Claridades Revista de Filosofía · doi:10.24310/crf.16.2.2024.19629 · Cited by 1
Competencies Replaceable by Artificial Intelligence in the Tuning Project for Latin America
Romero-Sandoval, A. (2024) · Artificial intelligence · doi:10.5772/intechopen.1008225 · Cited by 2
From principles to practice: The case for coordinated international LLMs supervision
Borgogno, O., & Perrazzelli, A. (2025) · Cambridge Forum on AI Law and Governance · doi:10.1017/cfl.2024.13 · Cited by 1
Against Purposeful Artificial Intelligence Failures
Yampolskiy, R. V. (2024) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v1i1.9943 · Cited by 1
Exploring The Nexus Between The Use of Artificial Intelligence in Communication Management
Zamrudi, Z., Mardiana, M., Khuzaini, K., & Lihu, D. C. K. (2024) · IJPGM International Journal of Public Governance and Management · doi:10.31602/ijpgm.v1i1.17097 · Cited by 2
“This progression shows a trend in the field, moving from purely technological advancements to addressing the broader implications of AI on human and organizational dynamics (Rohde et al., 2023; Bengio et al., 2024).”
Li, X., Xiang, H., & Zhou, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92712-6_26 · Cited by 1
Judijanto, L. (2025) · Science of law. · doi:10.55284/4r3qs710 · Cited by 1
“The lag between technological development and policy response creates a vulnerability window where high-risk AI applications may operate with minimal scrutiny (Bengio et al., 2024).”
Using AI and Concerns in Corporate Learning
Kubovics, M. (2025) · International Journal of Business Law and Education · doi:10.56442/ijble.v6i2.1126 · Cited by 1
First-Person Plural (Versus Singular) Pronoun Use Is Linked to Greater Perception of AI Threat
Li, C., Dang, J., & Liu, L. (2025) · Personality and Social Psychology Bulletin · doi:10.1177/01461672251356107 · Cited by 1
“Despite the impressive capabilities of AI, concerns about its potential pitfalls—ranging from economic displacement to doomsday scenarios—continue to be voiced (e.g., Bengio et al., 2024; Fung, 2023).”
Challenges for Foundation Model Liability and Regulatory Regimes: An Analysis of US Law
Henderson, P. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009492553.012 · Cited by 1
Engineering U.S. Responsible AI Policy, A Survey, 2020-2025
Byrne, D. (2025) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--55292 · Cited by 3
Atoum, I. A., & Altahat, S. (2025) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v14.i4.pp2655-2666 · Cited by 4
Ten pillars of AI-enabled shared decision making: A practical guide
Rahimi, S. A., Poenaru, D., Elwyn, G. J., & Prévost, K. (2025) · Oxford University Press eBooks · doi:10.1093/med/9780198897804.003.0061 · Cited by 1
The Symmetry of Interdependence in Human–AI Teams and the Limits of Classical Team Science
Lawless, W. F. (2025) · AppliedMath · doi:10.3390/appliedmath5030114 · Cited by 1
“In particular, with AI, one claim today is that AI is potentially dangerous (e.g., [3]), especially if AI gains agency [4].”
López, M. Á., & Petrelli, M. (2025) · Artificial Intelligence in Geosciences · doi:10.1016/j.aiig.2025.100153 · Cited by 3
Lo, H. F. J. (2025) · Springer eBooks · doi:10.1007/978-3-031-97999-6_13 · Cited by 1
Ballini, F., Lee, S.-Y., & Lee, E.-K. (2025) · World Maritime University eBooks · doi:10.21677/250911 · Cited by 1
AI Consciousness and Existential Risk
VanRullen, R. (2026) · Journal of Artificial Intelligence and Consciousness · doi:10.1142/s2705078526500049 · Cited by 1
AI deployment for heterogeneous ports: Strategic choices between independent R&D and outsourcing
Lu, B., Zhang, G., & Li, Y. (2025) · Transport Policy · doi:10.1016/j.tranpol.2025.103977 · Cited by 1
Assessing the Relational Abilities of Large Language Models and Large Reasoning Models
Raemaekers, M., Finn, M., & Houwer, J. D. (2025) · Behavioral Sciences · doi:10.3390/bs16010045 · Cited by 1
Jailbreaking LLMs Through Tense Manipulation in Multi-turn Dialogues
Tchuindjang, M., Duran, N., Legg, P., & Medjek, F. (2026) · Advances in intelligent systems and computing · doi:10.1007/978-3-032-07938-1_24 · Cited by 2
Shan, W., Fu, M. C., Yang, R., & Tantithamthavorn, C. K. (2025) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1109/aiware69974.2025.00029 · Cited by 4
“Ensuring their reliability is a unique research challenge, as noted by Bengio et al. [7], Hassan et al. [8], and Yao et al. [9], since quality assurance must address the open-ended, unpredictable nature of model outputs.”
The Polish adaptation of the artificial intelligence attitude scale: AIAS-4-PL
Juranek, J. F., & Pochwatko, G. (2026) · Current Psychology · doi:10.1007/s12144-025-08930-5 · Cited by 2
“AI development brings potentially many societal chal - lenges and risks which could be expanded with the occur - rence of AGI (Baum, 2017 ; Bengio et al., 2024; McLean et al., 2023 ).”
Chu, K. M., & Chan, H. C. (2026) · AI and Ethics · doi:10.1007/s43681-026-00989-5 · Cited by 1
On the inherent robustness of one-stage object detection against out-of-distribution data
Martínez-Seras, A., Ser, J. D., Olivares-Rad, A., Andrés, A., & Bringas, P. G. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.108683 · Cited by 1
On the origin of Earth’s Universal BioConsciousness
Tleuberdinova, A. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1636473 · Cited by 2
“…the best-selling author of the books “Nexus” & “Human Sapiens,” and Global AI Safety Initiative founder and advocate Yoshua Bengio warn the global community about the novel risks to humanity posed by AI’s rapid progress, which can exacerbate global injustice and inequity (Bengio et al., 2024).”
Blame and Cost: Understanding Responsibility Attribution and Financial Risk in AI Usage
Fildor, D., Škrinjarić, B., & Budak, J. (2026) · Journal of Contingencies and Crisis Management · doi:10.1111/1468-5973.70124 · Cited by 1
Risk Management Framework for Procuring AI Solutions
Miller, C. L. (2026) · American Journal of Management · doi:10.33423/ajm.v26i1.8098 · Cited by 1
Wang, B., Bai, Y., & Li, J. (2026) · International Conference on Computer Technology Applications (ICCTA) · doi:10.1109/iccta70480.2026.11706424 · Cited by 1
“Furthermore, as highlighted by Bengio et al. [6] and Kasneci et al. [7], generic models often hallucinate or lack domain-specific grounding.”
Can Artificial Intelligence Destroy Future Industry?
Terziyan, V. Y., Gryshko, S., Kaikova, O., & Golovianko, M. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.02.338 · Cited by 1
Ethics, Bias Mitigation, and Model Risk Management
Parween, S., Dubey, N., Bharadwaj, R., Reddy, B. A., Sanghavi, H., Prasan, U. D., & Bhoopathy, V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6851-1.ch006 · Cited by 1
Janson, N., Essex, A. E., & Balanov, A. G. (2026) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2026.135226 · Cited by 1
MedChat: a fully offline multimodal AI system for privacy-preserving clinical anamnesis
Ruhland, J. B., Bahcivan, D., Sowa, J., Canbay, A. E., & Heider, D. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1809142 · Cited by 1
“Artificial intelligence (AI) has reached a pivotal moment in its development, leading to transformative changes across nearly all sectors of human society (Jarrahi, 2018; Sarumi and Heider, 2024; Shi and Sun, 2024; Bengio et al., 2024).”
Adversarial Robustness of Discriminative Self-Supervised Learning in Vision
Çağatan, Ö. V., Tal, O. F., & Gürsoy, M. E. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00223 · Cited by 4
“As these models become increasingly widespread and integrated into various applications, ensuring their reliability and safety has become a critical concern [6, 7].”
Xu, X., Feng, S., Shen, G., & Zhang, X. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813176 · Cited by 1
Serracín, J., & Gómez, J. (2026) · Revista Científica de Salud y Desarrollo Humano · doi:10.61368/r.s.d.h.v7i2.1047 · Cited by 1
Educating Managers to Govern Artificial Intelligence
Osadchyi, V., Shantyr, A., Zinchenko, O., Bondarchuk, A., Лащевська, Н. О., & Osadchа, K. (2026) · Sustainability · doi:10.3390/su18115590 · Cited by 1
Geist, E., Meyer, A. D., Moon, A., Najera, A., Jones, J. H., & Wu, A. (2026) · npj Complexity · doi:10.1038/s44260-026-00090-2 · Cited by 1
“And concerns about potential harms from AI have led to calls to better understand the capabilities of AI and to regulate AI development 5 .”
Jimenez, J. C., McIntosh, D. F., Rowe, V. L., & Hines, O. J. (2026) · JVS-Vascular Insights · doi:10.1016/j.jvsvi.2026.100486 · Cited by 1
Palminteri, S., & Pistilli, G. (2026) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2026.101670 · Cited by 1
Realigning AI technology towards the Sustainable Development Goals
Feng, R., Li, Z., Fu, B., Guo, H., Liu, J., & Bengio, Y. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01271-3 · Cited by 1
Self-citationIs it safe to deploy AI in safety-critical systems?
Kolda, T. G. (2026) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2025.0123 · Cited by 2
Perceived control influences users' acceptance of AI-enabled services
Li, Z., & Li, H. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107512 · Cited by 1
Value creation and capture in artificial intelligence‐enabled ecosystems
Kapoor, R., & Raj, M. (2026) · Strategic Entrepreneurship Journal · doi:10.1002/sej.70057 · Cited by 7
Šķilters, J., Pokrotnieks, J., & Derovs, A. (2024) · Proceedings of the Latvian Academy of Sciences Section B Natural Exact and Applied Sciences · doi:10.2478/prolas-2024-0032 · Cited by 1
Introduction to Explainable AI: Unveiling the Black Box
Mittal, D., Parashar, A. R., & Singh, P. K. (2024) · Advances in environmental engineering and green technologies book series · doi:10.4018/979-8-3693-7822-9.ch006 · Cited by 1
The New Frontiers of Tech Companies in the Age of AI
Girgin, D. (2024) · Human computer interaction. · doi:10.62802/jdcc3w50 · Cited by 1
Gude, Μ., Weck, D., & Müller-Pabel, M. (2024) · doi:10.25368/2024.52 · Cited by 1
Employment of AI in Higher Education
Jain, R., Parmar, K. J., Palaniappan, D., & Premavathi, T. (2025) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-7949-3.ch003 · Cited by 2
Ієрархічне та мережеве суспільство: антагонізм чи субсидіарність?
Пасько, Я. І. (2024) · Актуальні проблеми філософії та соціології · doi:10.32782/apfs.v048.2024.40 · Cited by 1
The Artificial Intelligence Paradox: Among Global Power, Errors, and Regulatory Shifts
Baldoni, R. (2025) · IEEE Security & Privacy · doi:10.1109/msec.2025.3550927 · Cited by 1
Follow the money, not the majority: A mechanism predicting unresolvable events
Baillon, A., Tereick, B., & Wang, T. V. (2025) · Journal of Risk and Uncertainty · doi:10.1007/s11166-025-09463-9 · Cited by 1
“They also widely disagree: some subject-matter experts believe that the risks are substantial (Yudkowsky et al., 2008; Bostrom, 2014; Russell, 2019), and possibly imminent (Bengio et al., 2024; Kokotajlo et al., 2025); others are more skeptical (Thorstad, 2024; Narayanan & Kapoor, 2024).”
Debates and Disruptions in the Global Governance of Education: Taking Stock in Tumultuous Times
Mundy, K., Menashy, F., Jules, T. D., & Salajan, F. D. (2025) · Comparative Education Review · doi:10.1086/736502 · Cited by 1
Defending Democracy and DemocraciesPreprint
Michaels, R. A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4920299 · Cited by 1
Zwass, V. (2024) · Journal of Management Information Systems · doi:10.1080/07421222.2024.2376369 · Cited by 1
Digital Homunculi and Institutional Design: Breaking Through the Experimentation BottleneckPreprint
Špecián, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.00826 · Cited by 1
“Nowadays, advanced AI systems are swiftly arriving on the scene, threatening ‘infocalypse,’ labor market upheaval, or worse (Schick 2020; Bengio et al. 2024).”
SAGE: Scalable Ground Truth Evaluations for Large Sparse AutoencodersPreprint
Venhoff, C., Calinescu, A., Torr, P. H. S., & de Witt, C. S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.07456 · Cited by 3
“However, the deployment of LLMs in such critical areas raises significant safety and ethical concerns (Bengio et al., 2024; Anderljung et al., 2023; Hendrycks & Mazeika, 2022), as their decisions can have profound consequences.”
BackgroundSelf-citationInsuring Uninsurable Risks from AI: Government as Insurer of Last ResortPreprint
Trout, C. (2024) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2409.06672 · Cited by 4
“Many experts believe AI systems will, sooner or later, pose uninsurable risks, including existential risks (Grace et al., 2024; Bengio et al., 2024).”
Online Influence Campaigns: Strategies and VulnerabilitiesPreprint
Musulan, A., Xia, V., Kosak-Hine, E., Gibbs, T., Sujaya, V., Rabbany, R., Godbout, J., & Pelrine, K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5106352 · Cited by 1
Do AI Laws Inhibit Innovation?Preprint
Ciconte, W., Rozario, A., & Urcan, O. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5046045 · Cited by 2
Jiang, L., Chen, J., Lu, L., Peng, X., Liu, L., He, J.-J., & Xu, J. (2025) · doi:10.2196/preprints.80049 · Cited by 2
Metadata
- Authors
- Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Darrell, T., Harari, Y. N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., Hadfield, G., Clune, J., Maharaj, T., Hutter, F., Baydin, A. G., McIlraith, S., Gao, Q., Acharya, A., Krueger, D., Dragan, A., Torr, P., Russell, S., Kahneman, D., Brauner, J., & Mindermann, S.
- Affiliations
- Mila - Quebec Artificial Intelligence Institute (CA)
- University of Toronto (CA)
- Tsinghua University (CN)
- University of California, Berkeley (US)
- Hebrew University of Jerusalem (IL)
- Institute for AI International Governance, Tsinghua University, Beijing, China.
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel.
- University of British Columbia (CA)
- University of Freiburg (DE)
- University of Oxford (GB)
- East China University of Political Science and Law (CN)
- RAND Corporation (US)
- University of Cambridge (GB)
- Princeton University (US)
- ORCID
- Venue
- Science
- Published
- 2024-05-24
- Volume
- 384
- Issue
- 6698
- Pages
- 842-845
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4398133596
- Cited by
- 349
- References
- 3
- Citations by year
- 2026: 125 · 2025: 161 · 2024: 59
- FWCI
- 142.99
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- AI risks, AI governance, risk management, proactive governance, AI safety
- MeSH terms
- COVID-19, Artificial Intelligence, Humans, Risk Management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.