Managing extreme AI risks amid rapid progress
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Preparation requires technical research and development, as well as adaptive, proactive governance
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Any advancement that increases the autonomous capabilities of AI systems introduces its own set of safety considerations (Bengio et al., 2024), especially for systems that improve in an open-ended way (Ecoffet et al., 2020; Clune, 2019).”
From Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents (2025) · cites this paper as Bengio et al., 2024
In the citing paper
“Advanced AI also introduces phenomena that differ fundamentally from previous generations of AI or other technologies, requiring new approaches to mitigating these risks (Bengio et al., 2024).”
From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Bengio et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationA Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925 · cited 2×
“However, AI is making progress in critical domains such as hacking, social manipulation, and strategic planning 24,31 , and may soon pose unprecedented con-trol challenges.”
BackgroundEmergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Augmenting large language models with chemistry tools
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00832-8 · Cited by 794
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“Especially in the hands of a few powerful actors, AI could cement or exacerbate global inequities, or facilitate automated warfare, customized mass manipulation, and pervasive surveillance 18–23 .”
BackgroundA graph placement methodology for fast chip design
Mirhoseini, A., Goldie, A., Yazgan, M., Jiang, J. W., Songhori, E., Wang, S., Lee, Y.-J., Johnson, E., Pathak, O., Nova, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03544-w · Cited by 618 · cited 1×
“Progress in AI also enables faster AI progress 8 : AI assistants are increasingly used to automate programming 9 , data collection 10,11 , and chip design 12 .”
MethodSuperhuman AI for multiplayer poker
Brown, N., & Sandholm, T. (2019) · Science · doi:10.1126/science.aay2400 · Cited by 592
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.17580 · Cited by 477
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1657 · Cited by 475
Research works printed but not linked: 8
- M. Kinniment et al. arXiv:2312.11671 (18 December 2023). · arXiv
- I. R.McKenzie. arXiv:2306.09479 (15 June 2023). · arXiv
- R.Ngo L.Chan S.Mindermann arXiv:2209.00626 (20 August 2022). · arXiv
- E.Hubinger. arXiv:2401.05566 (10 January 2024). · arXiv
- L.Koessler J.Schuett arXiv:2307.08823 (17 July 2023). · arXiv
- N. Kolt et al. arXiv:2404.02675 (3 April 2024). · arXiv
- M. Phuong et al. arXiv:2403.13793 (20 March 2024). · arXiv
- T. A. Kelly, SAE Trans. J. Mater. Manuf. 113, 257 (2004).
Cited by
Items 201–300 of 618
China’s emerging regulation toward an open future for AI
Zhu, Y. F., He, B., Fu, H., Hu, N., Wu, S., Zhang, T., Liu, X., Xu, G., Zhang, L., & Zhou, H. (2025) · Science · doi:10.1126/science.ady7922 · Cited by 6
Defending Against AI Threats with a User-Centric Trustworthiness Assessment Framework
Kafali, E., Preuveneers, D., Semertzidis, T., & Daras, P. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8110142 · Cited by 7
Goldstein, J. A., & Sastry, G. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31653 · Cited by 8
Towards Shutdownable Agents via Stochastic Choice
Thornley, E., Roman, A., Ziakas, C., Ho, L., & Thomson, L. (2024) · Transactions on machine learning research · doi:10.48550/arxiv.2407.00805 · Cited by 7
Surprising gender biases in GPTPreprint
Fulgu, R. A., & Capraro, V. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4888393 · Cited by 9
AI threats to national security can be countered through an incident regimePreprint
Ortega, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.19887 · Cited by 9
“In this scenario, a misaligned, highly capable AI system could pose a threat to national security by e.g. creating bioweapons or executing a large-scale cyber attack (Bengio et al., 2024; Hendrycks et al., 2023).”
Jalilian, L., McDuff, D., & Kadambi, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.16902 · Cited by 9
Artificial Intelligence and Emerging Risks in Occupational Safety and Health
Baraza, X., & Torrent‐Sellens, J. (2026) · Encyclopedia · doi:10.3390/encyclopedia6010025 · Cited by 8
Training Agents to Self-Report MisbehaviorPreprint
Lee, B. W., Yueh-Han, C., & Korbak, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.22303 · Cited by 8
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025) · arXiv · doi:10.48550/arxiv.2505.22954 · Cited by 344
Self-citationThe state as a model for AI control and alignment
Elsner, M. (2024) · AI & Society · doi:10.1007/s00146-024-02063-2 · Cited by 7
“Some proposals have come from researchers (Hacker et al. 2023; Bengio et al. 2024).”
Lee, Y., Chang, S., Kim, J., & Kim, I. (2024) · Journal of Marine Science and Engineering · doi:10.3390/jmse12122212 · Cited by 5
AI Alignment Versus AI Ethical Treatment: 10 Challenges
Bradley, A., & Saad, B. (2025) · Analytic Philosophy · doi:10.1111/phib.12380 · Cited by 7
Machines in the crowd? Measuring the footprint of machine-generated text on Reddit
Cava, L. L., Aiello, L. M., & Tagarelli, A. (2026) · Online Social Networks and Media · doi:10.1016/j.osnem.2026.100355 · Cited by 5
Yao, L., Gu, T., Li, X., Jiao, Y., Li, M., Graff, J. C., Li, Y., & Gu, W. (2025) · Die Naturwissenschaften · doi:10.1007/s00114-025-02036-5 · Cited by 6
“However, spreading misinformation about retracted articles by AI models harms the medical research as well as the clinical practice (Bengio et al. 2024).”
Park, J.-A., Kim, C. H., & Kim, H.-J. (2025) · Telecommunications Policy · doi:10.1016/j.telpol.2025.103106 · Cited by 9
Evaluating LLMs in Open-Source Games
Sistla, S., & Kleiman-Weiner, M. (2025) · neural information processing systems · doi:10.52202/085713-3474 · Cited by 7
Digital Divide (2.0): the Shadow of AI Technology
Kettani, D., Moulin, B., & Barbosa, L. S. (2025) · Perspectives on Global Development and Technology · doi:10.1163/15691497-12341720 · Cited by 5
Li, D., Li, M., Wang, Z., & Li, F. (2026) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2026.112363 · Cited by 9
Beyond computational equivalence: the behavioral inference principle for machine consciousness
Palminteri, S., & Wu, C. M. (2026) · Neuroscience of Consciousness · doi:10.1093/nc/niag002 · Cited by 7
“The first ethical question is related to the problem of control and potential existential harm that extremely powerful artificial agents can cause to the human race (Bengio et al., 2023) .”
Herrera, F., García, S., del Jesús, M. J., Sánchez, L., & de Prado, M. L. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8030069 · Cited by 8
“Bengio et al. (2024) [14] and Hendrycks et al. (2023) [15] emphasize existential and catastrophic risks, such as runaway misalignment or misuse of bioterrorism, calling for anticipatory regulation and frontier system testing.”
Zhang, Z., & Valeo, C. (2026) · AI for Engineering · doi:10.3390/aieng1010005 · Cited by 6
“For example, should shell permissions be granted to the run-time [66,67]?”
NeurIPS should lead scientific consensus on AI policy
Bommasani, R. (2025) · neural information processing systems · doi:10.52202/085713-1197 · Cited by 7
Visual analysis of drug research and development based on artificial intelligence
Wei, W., Song, C., Qi, C., Zhang, X., Zhang, X., Pu, R., & Ao, Y. (2024) · Journal of Holistic Integrative Pharmacy · doi:10.1016/j.jhip.2024.12.002 · Cited by 6
The Sentient AI Cloud: A Conscious Digital Mind Governing the Internet
AAMC, W., & Talati, D. V. (2024) · International Journal of Innovative Research in Computer and Communication Engineering · doi:10.15680/ijircce.2024.1209001 · Cited by 5
PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations
Li, J., Hu, R., Huang, K., Zhuang, Y., Liu, Q., Zhu, M., Shi, X., & Wei, L. (2024) · neural information processing systems · doi:10.52202/079017-0341 · Cited by 18
SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
Feng, K., Ding, K., Wang, Y., Li, M., Wei, F., Wang, X., & Chen, H. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1808 · Cited by 5
Artificial Intelligence and the Discovery of New Ideas: Is an Economic Growth Explosion Imminent?Preprint
Almeida, D., Naudé, W. A., & Sequeira, T. N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4709699 · Cited by 17
Crying wolf: Warning about societal risks can be reputationally riskyPreprint
Caviola, L., Coleman, M. B., Winter, C. K., & Lewis, J. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/gtr53 · Cited by 6
Surprising gender biases in GPTPreprint
Fulgu, R. A., & Capraro, V. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/mp27q · Cited by 6
Examining Popular Arguments Against AI Existential Risk: A Philosophical AnalysisPreprint
Swoboda, T., Uuk, R., Lauwaert, L., Rebera, A. P., Oimann, A., Chomanski, B., & Prunkl, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.04064 · Cited by 5
“Discussions of AI as an extinction risk to humanity generally revolve around two key pathways (Bengio et al., 2024; Bostrom, 2014; Carlsmith, 2024; Hendrycks et al., 2023; Russell, 2019).”
Can We Standardise the Frontier of AI?Preprint
Roberts, H., & Ziosi, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5271446 · Cited by 7
Shutdownable Agents through POST-AgencyPreprint
Thornley, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.20203 · Cited by 6
Course-Correction: Safety Alignment Using Synthetic PreferencesPreprint
Xu, R., Cai, Y., Zhou, Z., Gu, R., Weng, H., Liu, Y., Zhang, T., Xu, W., & Qiu, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.16637 · Cited by 6
Faldor, M., Zhang, J., Cully, A., & Clune, J. (2024) · arXiv · doi:10.48550/arxiv.2405.15568 · Cited by 4
Self-citationGeneral AI May Revolutionize Neurology—Or It Might Be Bad
Westover, M. B., & Westover, A. M. (2025) · JAMA Neurology · doi:10.1001/jamaneurol.2025.0905 · Cited by 4
A value-driven system design framework for sustainable 6G networks
Sciddurlo, G., de Trizio, F., Piro, G., & Boggia, G. (2025) · Computer Networks · doi:10.1016/j.comnet.2025.111477 · Cited by 4
Application of artificial intelligence in the development of circular bioeconomy
Mishra, R., & Pandey, D. K. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-26779-6.00014-0 · Cited by 4
Wasil, A. R., Chen, M., & Turner, J. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_57 · Cited by 4
Send charcoal in snowy weather: Artificial intelligence and organizational resilience
Guo, T., Shang, J., & Ding, X. (2026) · Asia Pacific Journal of Management · doi:10.1007/s10490-025-10107-4 · Cited by 7
AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software
Yang, R., Fu, M. C., Tantithamthavorn, C. K., Arora, C., Gulmammadova, G., & Chua, J. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00279 · Cited by 7
“Importantly, this challenge observed in practice mirrors concerns raised in recent literature on the security and safety of LLM agents [3]–[5], [21].”
Toward AI-pervasive electric energy systems: Concept, framework, vulnerability, and visions
Ruan, J., Xu, Z., Sun, X., Ding, Y., Zhao, J., & Dong, Z. Y. (2026) · Nexus · doi:10.1016/j.ynexs.2026.100119 · Cited by 4
AI, trust, and the market for lemons: rethinking the credibility of entrepreneurship research
Obschonka, M., & Lévesque, M. (2026) · Small Business Economics · doi:10.1007/s11187-026-01180-0 · Cited by 4
Navigating artificial intelligence narratives in Chinese media: a mixed-methods discourse analysis
Li, Y., Yu, D., & Yu, L. (2026) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqag004 · Cited by 4
Multimodal AI needs active human interaction
Pan, X., & Schwartz, O. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01932-6 · Cited by 4
Isles of autonomy: the rise of intelligent technologies
Hancock, P. A. (2025) · Ergonomics · doi:10.1080/00140139.2024.2447863 · Cited by 4
“Those who disagree with such an assessment confuse enthusiastic pessimism with prospective reality (Bengio et al. 2024).”
Belief updating in AI‐risk debates: Exploring the limits of adversarial collaboration
Rosenberg, J., Karger, E., Jacobs, Z., Hickman, M. G., Morris, A., Durland, H., Kuusela, O., & Tetlock, P. E. (2025) · Risk Analysis · doi:10.1111/risa.70023 · Cited by 6
“We report two studies that investigate the potential of adversarial collaborations in debates over the risks to humanity from advanced AI, a topic that has drawn significant concern from prominent thinkers in the AI field (Bengio et al., 2024).”
Gutiérrez, E. J. D., & Jarquín-Ramírez, M.-R. (2025) · Sintaxis · doi:10.36105/stx.2025n14.04 · Cited by 5
From the Pascal Wager to Value Engineering: A Glance at AI Risks and How to Address Them
Noriega, P., & Casanovas, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-85463-7_16 · Cited by 6
“8 5 There is a lively ongoing discussion about the interplay between risks, autonomy, values and AI that this paper addresses; for example, [32,40,22,4].”
On the Consideration of AI Openness: Can Good Intent Be Abused?
Kim, Y., Shin, H., Choi, E., Oh, H., Kim, H., & Hwang, W. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i26.34948 · Cited by 6
Humanity Amplified: The Fusion of Deep Learning and Human Insight to Shape the Future of InnovationPreprint
Bauschard, S., & Quidwai, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4621210 · Cited by 6
Safeguarding the safeguards: How best to promote AI alignment in the public interestPreprint
Guest, O., Aird, M., & hÉigeartaigh, S. Ó. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.08039 · Cited by 7
An Entropy Approach to Interdependent Human–Machine Teams
Lawless, W. F., & Moskowitz, I. S. (2025) · Entropy · doi:10.3390/e27020176 · Cited by 7
Bridging Today and the Future of Humanity: AI Safety in 2024 and BeyondPreprint
Han, S. (2025) · Qeios · doi:10.32388/zhm9uq · Cited by 5
The impacts of artificial intelligence on environmental sustainability and human well-beingPreprint
Carmeno, N. L., Domingos, T., & O'Neill, D. W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.24091 · Cited by 6
Can LLMs make trade-offs involving stipulated pain and pleasure states?
Keeling, G., Street, W., Stachaczyk, M., Zakharova, D., Comsa, I. M., Sakovych, A., Logothetis, I., Zhang, Z., Arcas, B. A. Y., & Birch, J. (2024) · arXiv · doi:10.48550/arxiv.2411.02432 · Cited by 3
“…malicious actors to engage in harmful forms of misuse, alongside agentic capabilities such as self-reasoning and autonomous self-replication that could enable models to realise harmful outcomes (Phuong et al., 2024; Bengio et al., 2024; Guest et al., 2024; Anthropic, 2023b; Kinniment et al., 2024).”
BackgroundAI Assistance Reduces Persistence and Hurts Independent Performance
Liu, G., Christian, B., Dumbalska, T., Bakker, M. A., & Dubey, R. (2026) · arXiv · doi:10.48550/arxiv.2604.04721 · Cited by 3
“AI safety research has traditionally focused on abrupt, dire events caused by advanced AI systems (Bucknall & Dori-Hacohen, 2022; Carlsmith, 2022; Bengio et al., 2024).”
Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agents
Buscemi, A., Proverbio, D., Bova, P., Balabanova, N., Bashir, A., Cimpeanu, T., da Fonseca, H. C., Duong, M. H., Domingos, E. F., Fernandes, A. M., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.08640 · Cited by 3
“As Artificial Intelligence (AI) applications become widespread, debates are taking place about how to regulate it [1–6].”
AI Safety for Physical Infrastructures: A Collaborative and Interdisciplinary Approach
Farahmand, F., & Neu, R. W. (2025) · Fatigue & Fracture of Engineering Materials & Structures · doi:10.1111/ffe.14575 · Cited by 5
Artificial Intelligence in Mental Healthcare: Opportunities, Risks, and Regulations
Jafar, A., Shabnam, S., & Khan, A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5072-1.ch014 · Cited by 3
AI Technologies for Improving Education Systems
Ayyaswamy, K. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-3236-9.ch003 · Cited by 3
Optimizing All-to-All Collective Communication with Fault Tolerance on Torus Networks
Qin, L., Cui, J., Cai, W., Niu, M., Yang, Y., & Huang, J. (2025) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3725843.3756057 · Cited by 3
Kuang, J., Xie, J., & Yan, Z. (2025) · MIS Quarterly · doi:10.25300/misq/2025/18897 · Cited by 3
Product Safety in the Age of AI: Autonomy, R&D, and Liability
Chen, Y., & Hua, X. (2026) · The Economic Journal · doi:10.1093/ej/ueag036 · Cited by 7
Responsible artificial intelligence integration framework for psychiatric guidelines
Mendlovic, S., Frankova, I., Vermetten, E. H. G. J. M., Wasserman, D., Schulze, T. G., Falkai, P., Fountoulakis, K. Ν., Adorjan, K., Uchida, H., Fruchter, E., et al. (2026) · The International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyag010 · Cited by 4
Reznikova, N. V., Panchenko, V., & Грод, М. І. (2026) · ACTUAL PROBLEMS OF INTERNATIONAL RELATIONS · doi:10.17721/apmv.2026.166.1.147-163 · Cited by 3
Legal infrastructure for transformative AI governance
Hadfield, G. K. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2509742123 · Cited by 5
Self-citationCourse-Correction: Safety Alignment Using Synthetic Preferences
Xu, R., Cai, Y., Zhou, Z., Gu, R., Weng, H., Yan, L., Zhang, T., Xu, W., & Qiu, H. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-industry.119 · Cited by 13
Pande, A., Kumar, A., & Anjankar, A. P. (2025) · Multidisciplinary Reviews · doi:10.31893/multirev.2025218 · Cited by 3
“It lists the FDA’s framework for Software as a Medical Device and the European Union's General Data Protection Regulation as significant standards for how AI is utilised in healthcare (Bengio et al., 2024; Tang et al., 2023; Peters & Visser, 2023).”
Neural Reasoning for Sure Through Constructing Explainable Models
Dong, T., Jamnik, M., & Lió, P. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i11.33262 · Cited by 7
The superalignment of superhuman intelligence with large language models
Huang, M., Wang, Y., Cui, S., Ke, P., & Tang, J. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4348-6 · Cited by 4
AI-Powered Social Media for Development in Low- and Middle-Income Countries
Penh, B. (2025) · Media and Communication · doi:10.17645/mac.9577 · Cited by 3
Serkin, L., Paić, G., & Coello, C. A. C. (2024) · TIES Revista de Tecnología e Innovación en Educación Superior · doi:10.22201/dgtic.26832968e.2024.10.10 · Cited by 3
Almlie, G. S., Roaldsøy, E. Ø., Lande, I., & Heimdal, A. (2024) · International Conference on Engineering and Product Design Education (EPDE) · doi:10.35199/epde.2024.81 · Cited by 3
Dissecting Dissonance: Benchmarking Large Multimodal Models Against Self-Contradictory Instructions
Gao, J., Gan, L., Li, Y., Ye, Y., & Wang, D. Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72998-0_23 · Cited by 8
The AI Cloud: A Digital Intelligence Controlling the Web
AAMC, W., & Talati, D. V. (2024) · International Journal of Advanced Research in Education and Technology · doi:10.15680/ijarety.2024.1104002 · Cited by 3
Generative AI in Ransomware Evolution: Challenges and Countermeasures
Ishtaiwi, A. M., Alateef, S., & Alkasassbeh, M. S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0832-6.ch014 · Cited by 3
Human–machine teams: advantages afforded by the quantum-likeness of interdependence
Lawless, W. F., & Moskowitz, I. S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-340553-2.00024-1 · Cited by 3
León, M. (2025) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2025.200583 · Cited by 3
RetractedÉtat de la situation sur les impacts sociétaux de l'intelligence artificielle et du numérique - 2024
Obvia, Langlois, L., Cousineau, M., Gagnon, M.-P., Larnder, C. I., Naffi, N., Garneau, J., Lévesque, C., Brin, C., Guèvremont, V., et al. (2024) · doi:10.61737/cfrt3613 · Cited by 3
Artificial intelligence for neurodiversity
Li, L. Z., & Yang, P. (2024) · BMJ · doi:10.1136/bmj.q1384 · Cited by 3
Unlearning-based Neural InterpretationsPreprint
Choi, C. L., Duplessis, A., & Belongie, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.08069 · Cited by 4
Obschonka, M., & Lévesque, M. (2024) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2409.08890 · Cited by 6
Neural Interactive ProofsPreprint
Hammond, L., & Adam-Day, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.08897 · Cited by 4
How Malicious AI Swarms Can Threaten DemocracyPreprint
Schroeder, D. T., Cha, M., Baronchelli, A., Bostrom, N., Christakis, N. A., García, D., Goldenberg, A., Kyrychenko, Y., Leyton‐Brown, K., Lutz, N., et al. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/qm9yk_v2 · Cited by 4
Self-citationHow Malicious AI Swarms Can Threaten DemocracyPreprint
Schroeder, D. T., Cha, M., Baronchelli, A., Bostrom, N., Christakis, N. A., García, D., Goldenberg, A., Kyrychenko, Y., Leyton‐Brown, K., Lutz, N., et al. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/qm9yk_v1 · Cited by 3
Self-citationBreaking the Block: Preserving Data Continuity to Train Superior SAEs for Instruct ModelsPreprint
Li, J., Ye, H., Chen, Y., Li, X., Zhang, L., Alinejad-Rokny, H., Peng, J. C., & Yang, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.07691 · Cited by 4
“As these models increasingly permeate critical applications, understanding their internal mechanisms has become imperative for ensuring alignment and safety (Bengio et al., 2024; Bereska & Gavves, 2024; Ji et al., 2025).”
Puppets or partners? Governing cyborg propaganda in the digital public squarePreprint
Kunst, J. R., Bierwiaczonek, K., Cha, M., Ebrahimi, O. V., Fawcett-Atkinson, M., Følstad, A., Gollwitzer, A., Köbis, N., Marcus, G., Roozenbeek, J., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.13088 · Cited by 3
A pragmatic approach to regulating AI agentsPreprint
Hacker, P., & Holweg, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.22819 · Cited by 5
AI Finds A WayPreprint
Dharna, A., Lu, C., Sullivan, R., Lehman, J., Krakovna, V., & Clune, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.23875 · Cited by 3
Self-citationRisk Ceilings and Development Deadlines: Pacing AI under Uncertain Safety ProductivityPreprint
Gan, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7607580 · Cited by 3
Reciprocal trust and distrust in artificial intelligence systems: the hard problem of regulation
Maggetti, M. (2026) · AI & SOCIETY · doi:10.1007/s00146-026-02966-2 · Cited by 2
“…objectives explicitly specified by designers, meaning that AI systems can evolve and develop unforeseen patterns of trust and distrust through their own autonomous learning processes—patterns that their programmers neither explicitly specified nor necessarily anticipated (Bengio et al. 2024, 2023).”
Confidence Regulation Neurons in Language Models
Stolfo, A., Wu, B., Gurnee, W., Belinkov, Y., Song, X., Sachan, M., & Nanda, N. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3970 · Cited by 2
Agent-Supported Foresight for AI Systemic Risks: AI Agents for Breadth, Experts for Judgment
Fröhling, L., Giaconia, A., Bogucka, E. P., & Quercia, D. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790712 · Cited by 2
Olympians: humanity as a solution to the control problem for artificial superintelligence
McKay, D. (2025) · AI and Ethics · doi:10.1007/s43681-025-00712-w · Cited by 2
AI and Data Surveillance: Embedding a Human Rights-based Approach
Lee, T.-L., Sekalala, S. R., & Villarreal, P. A. (2025) · The Journal of Law Medicine & Ethics · doi:10.1017/jme.2025.17 · Cited by 4
“44 With its Constitutional mandate as the coordinating UN agency for public health, the WHO is well-positioned to issue guidelines, recommendations, or policy documents to harmonize or coordinate AI and data surveillance policies at the global level.”
A Field Programmable Gate Array‐Assisted Optoelectronic Emulator for Photonic Neuromorphic Computing
Li, J.-X., Hu, R., Wang, F., Shen, J., Cheng, Z., & Zhou, P. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202500223 · Cited by 2
Ethical Implications of AI-Driven Ethical Hacking: A Systematic Review and Governance Framework
Sufficient, H. M., Mohammed, A., & Danjuma, B. (2025) · Journal of Cyber Security · doi:10.32604/jcs.2025.066312 · Cited by 2
“Such cases align with [17] warning that AI autonomy without explainability risks opaque decision-making with irreversible consequences.”
Recent Advances in the Application of Artificial Intelligence in Alzheimer's Disease
Yao, L., Ni, J., Wei, M., Li, T., Li, F., Wang, T., Xiao, W., Shi, J., & Tian, J. (2025) · Current Alzheimer Research · doi:10.2174/0115672050410489250930175402 · Cited by 2
Sohail, M., Hongying, Z., Abdullah, M., Caraffini, F., Javed, A., & Eshkiki, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.115206 · Cited by 3
Metadata
- Authors
- Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Darrell, T., Harari, Y. N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., Hadfield, G., Clune, J., Maharaj, T., Hutter, F., Baydin, A. G., McIlraith, S., Gao, Q., Acharya, A., Krueger, D., Dragan, A., Torr, P., Russell, S., Kahneman, D., Brauner, J., & Mindermann, S.
- Affiliations
- Mila - Quebec Artificial Intelligence Institute (CA)
- University of Toronto (CA)
- Tsinghua University (CN)
- University of California, Berkeley (US)
- Hebrew University of Jerusalem (IL)
- Institute for AI International Governance, Tsinghua University, Beijing, China.
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel.
- University of British Columbia (CA)
- University of Freiburg (DE)
- University of Oxford (GB)
- East China University of Political Science and Law (CN)
- RAND Corporation (US)
- University of Cambridge (GB)
- Princeton University (US)
- ORCID
- Venue
- Science
- Published
- 2024-05-24
- Volume
- 384
- Issue
- 6698
- Pages
- 842-845
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4398133596
- Cited by
- 349
- References
- 3
- Citations by year
- 2026: 125 · 2025: 161 · 2024: 59
- FWCI
- 142.99
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- AI risks, AI governance, risk management, proactive governance, AI safety
- MeSH terms
- COVID-19, Artificial Intelligence, Humans, Risk Management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.