CoChat
Sign Up
PreprintCited by 77Ethics and Social Impacts of AIDigitalization, Law, and RegulationLaw, AI, and Intellectual Property

Frontier AI Regulation: Managing Emerging Risks to Public Safety

Open access
Anderljung, Markus, Barnhart, Joslyn, Korinek, Anton, Leung, Jade, O'Keefe, Cullen, Whittlestone, Jess, Avin, Shahar, Brundage, Miles +16 more·2023·arXiv
DOI 10.48550/arxiv.2307.03718arXiv 2307.03718
Open this paper in CoChat — free accountOpen PDF

Abstract

Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "frontier AI" models: highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilities can arise unexpectedly; it is difficult to robustly prevent a deployed model from being misused; and, it is difficult to stop a model's capabilities from proliferating broadly. To address these challenges, at least three building blocks for the regulation of frontier models are needed: (1) standard-setting processes to identify appropriate requirements for frontier AI developers, (2) registration and reporting requirements to provide regulators with visibility into frontier AI development processes, and (3) mechanisms to ensure compliance with safety standards for the development and deployment of frontier AI models. Industry self-regulation is an important first step. However, wider societal discussions and government intervention will be needed to create standards and to ensure compliance with them. We consider several options to this end, including granting enforcement powers to supervisory authorities and licensure regimes for frontier AI models. Finally, we propose an initial set of safety standards. These include conducting pre-deployment risk assessments; external scrutiny of model behavior; using risk assessments to inform deployment decisions; and monitoring and responding to new information about model capabilities and uses post-deployment. We hope this discussion contributes to the broader conversation on how to balance public safety risks and innovation benefits from advances at the frontier of AI development.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2307.03718

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Separation of duties prevents conflicts of interest, ensuring those who design generative models are not solely responsible for auditing them.

    “Separation of duties prevents conflicts of interest, ensuring those who design generative models are not solely responsible for auditing them [36,34].”

    From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [36]

    Section 4.2 recommends third-party audits and external scrutiny to make risk assessments more objective while complementing internal testing. This supports having people other than model developers involved in auditing, but the paper does not say that separation of duties prevents conflicts of interest; the claim states that effect more definitively than the source does. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Restrictions on the development or deployment of certain multi-agent systems could be a useful regulatory tool, but it remains to be seen what such restrictions should entail and whether/where they are feasible.

    “Restrictions on the development or deployment of certain multi-agent systems could be a useful regulatory tool (Anderljung et al., 2023; Mitchell et al., 2025), but it remains to be seen what such restrictions should entail and whether/where they are feasible.”

    From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Anderljung et al., 2023

    Section 3.3.3, “License Frontier AI Development and Deployment” (page unavailable), proposes licensing deployment of high-risk frontier AI models and potentially their development. That supports restrictions as a possible regulatory tool, but the passage does not specifically address multi-agent systems. The sentence’s later uncertainty about the form and feasibility of restrictions is not directly attributed to this citation. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Generative models and regulatory interpretations evolve.

    “generative models and regulatory interpretations evolve”

    From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [36]

  • Registries and rights offices maintain authoritative records for identifiers such as ISBN, ISWC, DOI, or platform-specific asset IDs.

    “Registries and rights offices maintain authoritative records for identifiers such as ISBN, ISWC, DOI, or platform-specific asset IDs”

    From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [36]

    Section 3.2 discusses regulators collecting information about frontier AI, but does not address registries or rights offices maintaining identifier records. Searches for ISBN, ISWC, DOI, platform-specific asset IDs, and authoritative registries found no passage supporting the claim. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

  • Separation of duties prevents conflicts of interest.

    “Separation of duties prevents conflicts of interest”

    From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [36]

    Section 4.2 recommends independent external scrutiny for objective risk assessment, and section 3.3.4 suggests rotating regulatory roles to mitigate regulatory capture. Neither passage says that separation of duties prevents conflicts of interest; searches for both terms found no passage addressing that claim. (LLM, conf=0.70)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • The Theory of Economic Regulation

    Stigler, G. J. (1971) · The Bell Journal of Economics and Management Science · doi:10.2307/3003160 · Cited by 8,098 · cited 1× · see citing works

    “Insufficient expertise increases the risk that information asymmetries between the regulated industry and regulators lead to regulatory capture [219], and reduce meaningful enforcement.”
    Background

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works

    “(20)See [56, 57, 45, 58, 59] However, there are tasks for which scaling leads to worse performance [60, 61, 62], though further scaling has overturned some of these findings, [36].”
    Background

  • Eight Things to Know about Large Language Models

    Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137 · cited 1× · see citing works

    “Reliably controlling powerful AI models’ behavior, in other words, remains a largely unsolved technical problem [19, 17, 92, 93, 65] and the subject of ongoing research.”
    Background

  • Model evaluation for extreme risks

    Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., Kokotajlo, D., Marchal, N., Anderljung, M., Kolt, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.15324 · Cited by 66 · cited 7× · see citing works

    “Evaluation results could be used to inform predictions of a models’ potential dangerous capabilities prior to training, allowing developers to intentionally steer clear of models with certain dangerous capabilities [25].”
    BackgroundMethodInfluentialSelf-citation

Flagged first, then by citations

Research works printed but not linked: 134

  1. Michael Moor et al. "Foundation models for generalist medical artificial intelligence". In: Nature 616.7956 (Apr. 2023), pp. 259-265. DOI: 10.1038/s41586-023-05881-4. · DOI · Paper page
  2. Peter Lee, Sebastien Bubeck, and Joseph Petro. "Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine". In: New England Journal of Medicine 388.13 (Mar. 2023). Ed. by Jeffrey M. Drazen, Isaac S. Kohane, and Tze-Yun Leong, pp. 1233-1239. DOI: 10.1056/nejmsr2214184. · DOI · Paper page
  3. Karan Singhal et al. Large Language Models Encode Clinical Knowledge. 2022. arXiv: 2212.13138 [cs.CL]. · arXiv
  4. Harsha Nori et al. Capabilities of GPT-4 on Medical Challenge Problems. 2023. arXiv: 2303.13375 [cs.CL]. · arXiv
  5. Drew Simshaw. "Access to A.I. Justice: Avoiding an Inequitable Two-Tiered System of Legal Services". In: SSRN Electronic Journal (2022).
  6. Yonathan A. Arbel and Shmuel I. Becher. "Contracts in the Age of Smart Readers". In: SSRN Electronic Journal (2020). DOI: 10.2139/ssrn.3740356. · DOI
  7. Noam Kolt. "Predicting Consumer Contracts". In: Berkeley Technology Law Journal 37.1 (2022).
  8. David Rolnick et al. Tackling Climate Change with Machine Learning. 2019. arXiv: 1906.05433 [cs.CY]. · arXiv
  9. Huseyin Tuna Erdinc et al. De-risking Carbon Capture and Sequestration with Explainable CO2 Leakage Detection in Time-lapse Seismic Monitoring Images. 2022. arXiv: 2212 . 08596 [physics.geo-ph].
  10. Priya L. Donti and J. Zico Kolter. "Machine Learning for Sustainable Energy Systems". In: Annual Review of Environment and Resources 46.1 (Oct. 2021), pp. 719-747. DOI: 10.1146/annurev- environ-020220-061831. · DOI
  11. Panagiota Galetsi, Korina Katsaliaki, and Sameer Kumar. "The medical and societal impact of big data analytics and artificial intelligence applications in combating pandemics: A review focused on Covid-19". In: Social Science & Medicine 301 (May 2022), p. 114973. DOI: 10.1016/j.socscimed. 2022.114973. · DOI
  12. David C. Danko et al. The Challenges and Opportunities in Creating an Early Warning System for Global Pandemics. 2023. arXiv: 2302.00863 [q-bio.QM]. · arXiv
  13. Rishi Bommasani et al. On the Opportunities and Risks of Foundation Models. 2022. arXiv: 2108. 07258 [cs.LG].
  14. Fabio Urbina et al. "Dual use of artificial-intelligence-powered drug discovery". In: Nature Machine Intelligence 4.3 (Mar. 2022), pp. 189-191. DOI: 10.1038/s42256-022-00465-9. · DOI · Paper page
  15. Richard Ngo, Lawrence Chan, and Sören Mindermann. The alignment problem from a deep learning perspective. 2023. arXiv: 2209.00626 [cs.AI]. · arXiv
  16. Michael K. Cohen, Marcus Hutter, and Michael A. Osborne. "Advanced artificial agents intervene in the provision of reward". In: AI Magazine 43.3 (Aug. 2022), pp. 282-293. DOI: 10.1002/aaai. 12064. · DOI · Paper page
  17. Dan Hendrycks et al. Unsolved Problems in ML Safety. 2022. arXiv: 2109.13916 [cs.LG]. · arXiv
  18. Dan Hendrycks and Mantas Mazeika. X-Risk Analysis for AI Research. 2022. arXiv: 2206.05862 [cs.CY]. · arXiv
  19. Joseph Carlsmith. Is Power-Seeking AI an Existential Risk? 2022. arXiv: 2206.13353 [cs.CY]. · arXiv
  20. Daniil A. Boiko, Robert MacKnight, and Gabe Gomes. Emergent autonomous scientific research capabilities of large language models. 2023. arXiv: 2304.05332 [physics.chem-ph]. · arXiv
  21. Eric Horvitz. On the Horizon: Interactive and Compositional Deepfakes. 2022. arXiv: 2209.01714 [cs.LG]. · arXiv
  22. Josh A. Goldstein et al. Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations. 2023. arXiv: 2301.04246 [cs.CY]. · arXiv
  23. Russell A Poldrack, Thomas Lu, and Gašper Beguš. AI-assisted coding: Experiments with GPT-4. 2023. arXiv: 2304.13187 [cs.AI]. · arXiv
  24. Tom B. Brown et al. Language Models are Few-Shot Learners. 2020. arXiv: 2005.14165 [cs.CL]. · arXiv
  25. OpenAI. GPT-4 Technical Report. 2023. arXiv: 2303.08774 [cs.CL]. · arXiv
  26. Aakanksha Chowdhery et al. PaLM: Scaling Language Modeling with Pathways. 2022. arXiv: 2204.02311 [cs.CL]. · arXiv
  27. Jean-Baptiste Alayrac et al. Flamingo: a Visual Language Model for Few-Shot Learning. 2022. arXiv: 2204.14198 [cs.CV]. · arXiv
  28. Emily M. Bender et al. "On the Dangers of Stochastic Parrots". In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. ACM, Mar. 2021. DOI: 10 . 1145 / 3442188.3445922. · DOI · Paper page
  29. Baobao Zhang et al. Forecasting AI Progress: Evidence from a Survey of Machine Learning Re- searchers. 2022. arXiv: 2206.04132 [cs.CY]. · arXiv
  30. Jordan Hoffmann et al. Training Compute-Optimal Large Language Models. 2022. arXiv: 2203. 15556 [cs.CL].
  31. Tom Henighan et al. Scaling Laws for Autoregressive Generative Modeling. 2020. arXiv: 2010.14701 [cs.LG]. · arXiv
  32. Pablo Villalobos. Scaling Laws Literature Review. 2023. URL: https://perma.cc/32GJ-FBGM.
  33. Joel Hestness et al. Deep Learning Scaling is Predictable, Empirically. 2017. arXiv: 1712.00409 [cs.LG]. · arXiv
  34. Ian R. McKenzie et al. Inverse Scaling: When Bigger Isn't Better. 2023. arXiv: 2306.09479 [cs.CL]. · arXiv
  35. Ethan Perez et al. Discovering Language Model Behaviors with Model-Written Evaluations. 2022. arXiv: 2212.09251 [cs.CL]. · arXiv
  36. Philipp Koralus and Vincent Wang-Maścianica. Humans in Humans Out: On GPT Converging Toward Common Sense in both Success and Failure. 2023. arXiv: 2303.17276 [cs.AI]. · arXiv
  37. Jason Wei et al. Emergent Abilities of Large Language Models. 2022. arXiv: 2206.07682 [cs.CL]. · arXiv
  38. Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are Emergent Abilities of Large Language Models a Mirage? 2023. arXiv: 2304.15004 [cs.AI]. · arXiv
  39. Steven C. H. Hoi et al. Online Learning: A Comprehensive Survey. 2018. arXiv: 1802 . 02871 [cs.LG].
  40. German I. Parisi et al. "Continual lifelong learning with neural networks: A review". In: Neural Networks 113 (May 2019), pp. 54-71. DOI: 10.1016/j.neunet.2019.01.012. · DOI · Paper page
  41. Jason Wei et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. 2023. arXiv: 2201.11903 [cs.CL]. · arXiv
  42. Timo Schick et al. Toolformer: Language Models Can Teach Themselves to Use Tools. 2023. arXiv: 2302.04761 [cs.CL]. · arXiv
  43. Tianle Cai et al. Large Language Models as Tool Makers. 2023. arXiv: 2305.17126 [cs.LG]. · arXiv
  44. Shehel Yoosuf and Yin Yang. "Fine-Grained Propaganda Detection with Fine-Tuned BERT". In: Pro- ceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda. Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 87-91. DOI: 10.18653/v1/D19-5011. URL: https://perma.cc/5CWN-HTU2. · DOI
  45. Takeshi Kojima et al. Large Language Models are Zero-Shot Reasoners. 2023. arXiv: 2205.11916 [cs.CL]. · arXiv
  46. Yongchao Zhou et al. Large Language Models Are Human-Level Prompt Engineers. 2023. arXiv: 2211.01910 [cs.LG]. · arXiv
  47. Imanol Schlag et al. Large Language Model Programs. 2023. arXiv: 2305.05364 [cs.LG]. · arXiv
  48. Alexander Matt Turner et al. Optimal Policies Tend to Seek Power. 2023. arXiv: 1912 . 01683 [cs.AI].
  49. Victoria Krakovna and Janos Kramar. Power-seeking can be probable and predictive for trained agents. 2023. arXiv: 2304.06528 [cs.AI]. · arXiv
  50. Evan Hubinger et al. Risks from Learned Optimization in Advanced Machine Learning Systems. 2021. arXiv: 1906.01820 [cs.AI]. · arXiv
  51. Dario Amodei et al. Concrete Problems in AI Safety. 2016. arXiv: 1606.06565 [cs.AI]. · arXiv
  52. Yotam Wolf et al. Fundamental Limitations of Alignment in Large Language Models. 2023. arXiv: 2304.11082 [cs.CL]. · arXiv
  53. Yuntao Bai et al. Constitutional AI: Harmlessness from AI Feedback. 2022. arXiv: 2212.08073 [cs.CL]. · arXiv
  54. Alexander Pan et al. Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark. 2023. arXiv: 2304.03279 [cs.LG]. · arXiv
  55. Markus Anderljung and Julian Hazell. Protecting Society from AI Misuse: When are Restrictions on Capabilities Warranted? 2023. arXiv: 2303.09377 [cs.AI]. · arXiv
  56. Jaime Sevilla et al. "Compute Trends Across Three Eras of Machine Learning". In: 2022 International Joint Conference on Neural Networks. 2022, pp. 1-8. DOI: 10.1109/IJCNN55064.2022.9891914. · DOI · Paper page
  57. Yu-Hui Chen et al. Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations. 2023. arXiv: 2304.11267 [cs.CV]. · arXiv
  58. Irene Solaiman et al. Release Strategies and the Social Impacts of Language Models. 2019. arXiv: 1908.09203 [cs.CL]. · arXiv
  59. Irene Solaiman. The Gradient of Generative AI Release: Methods and Considerations. 2023. arXiv: 2302.04844 [cs.CY]. · arXiv
  60. Toby Shevlane. Structured access: an emerging paradigm for safe AI deployment. 2022. arXiv: 2201.05159 [cs.AI]. · arXiv
  61. "How to be responsible in AI publication". In: Nature Machine Intelligence 3.5 (May 2021), pp. 367- 367. DOI: 10.1038/s42256-021-00355-6. · DOI
  62. Aviv Ovadya and Jess Whittlestone. Reducing malicious use of synthetic media research: Considera- tions and potential release practices for machine learning. 2019. arXiv: 1907.11274 [cs.CY]. · arXiv
  63. Wayne Xin Zhao et al. A Survey of Large Language Models. 2023. arXiv: 2303.18223 [cs.CL]. · arXiv
  64. Ben Buchanan. The Cybersecurity Dilemma: Hacking, Trust and Fear Between Nations. Oxford University Press, 2017.
  65. Tero Karras, Samuli Laine, and Timo Aila. A Style-Based Generator Architecture for Generative Adversarial Networks. 2019. arXiv: 1812.04948 [cs.NE]. · arXiv
  66. John Jumper et al. "Highly accurate protein structure prediction with AlphaFold". In: Nature 596.7873 (July 2021), pp. 583-589. DOI: 10.1038/s41586-021-03819-2. · DOI · Paper page
  67. Gustaf Ahdritz et al. "OpenFold: Retraining AlphaFold2 yields new insights into its learning mecha- nisms and capacity for generalization". In: bioRxiv (2022). DOI: 10.1101/2022.11.20.517210. URL: https://www.biorxiv.org/content/early/2022/11/22/2022.11.20.517210. · DOI
  68. Jack W. Rae et al. Scaling Language Models: Methods, Analysis & Insights from Training Gopher. 2022. arXiv: 2112.11446 [cs.CL]. · arXiv
  69. Arnav Gudibande et al. The False Promise of Imitating Proprietary LLMs. 2023. arXiv: 2305.15717 [cs.CL]. · arXiv
  70. Emily H. Soice et al. Can large language models democratize access to dual-use biotechnology? 2023. arXiv: 2306.03809 [cs.CY]. · arXiv
  71. Gillian K. Hadfield and Jack Clark. Regulatory Markets: The Future of AI Governance. 2023. arXiv: 2304.04914 [cs.AI]. · arXiv
  72. Matthew C. Stephenson. "Information Acquisition and Institutional Design". In: Harvard Law Review 124.4 (2011).
  73. Cary Coglianese, Richard Zeckhauser, and Edward A. Parson. "Seeking Truth for Power: Informa- tional Strategy and Regulatory PolicymakingPolicymaking". In: Michigan Law review 89.2 (2004), pp. 277-341.
  74. Thomas O. McGarity. "Regulatory Reform in the Reagan Era". In: Maryland Law Review 45.2 (1986).
  75. Rovy Van Loo. "Regulatory Monitors: Policing Firms in the Compliance Era". In: Columbia Law Review 119 (2019).
  76. Rovy Van Loo. "The Missing Regulatory State: Monitoring Businesses in an Age of Surveil- lanceSurveillance". In: Vanderbilt Law Review 72.5 (2019).
  77. Noam Kolt. "Algorithmic Black Swans". In: Washington University Law Review 101 (2023).
  78. Gary E. Marchant, Braden R. Allenby, and Joseph R. Herkert. The Growing Gap Between Emerging Technologies and Legal-Ethical Oversight. Springer, 2011. URL: https://perma.cc/4XXW-3RHH.
  79. Margaret Mitchell et al. "Model Cards for Model Reporting". In: Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, Jan. 2019. DOI: 10.1145/3287560.3287596. · DOI · Paper page
  80. Timnit Gebru et al. "Datasheets for datasets". In: 64.12 (2021), pp. 86-92.
  81. Thomas Krendl Gilbert et al. Reward Reports for Reinforcement Learning. 2023. arXiv: 2204.10817 [cs.LG]. · arXiv
  82. Jaime Sevilla, Anson Ho, and Tamay Besiroglu. "Please Report Your Compute". In: Communications of the ACM 66.5 (Apr. 2023), pp. 30-32. DOI: 10.1145/3563035. · DOI
  83. Inioluwa Deborah Raji et al. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. 2020. arXiv: 2001.00973 [cs.CY]. · arXiv
  84. Ann M. Lipton. "Not Everything Is About Investors: The Case for Mandatory Stakeholder Disclosure". In: Yale Journal on Regulation (). URL: https://perma.cc/G97G-3FL2.
  85. Jess Whittlestone and Jack Clark. Why and How Governments Should Monitor AI Development. 2021. arXiv: 2108.12427 [cs.CY]. · arXiv
  86. Jakob Mökander et al. Auditing large language models: a three-layered approach. 2023. arXiv: 2302.08500 [cs.CL]. · arXiv
  87. Inioluwa Deborah Raji et al. Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance. 2022. arXiv: 2206.04737 [cs.CY]. · arXiv
  88. Hannah Bloch-Wehba. "The Promise and Perils of Tech Whistleblowing". In: Northwestern University Law Review (Mar. 3, 2023).
  89. Jonas Schuett. Three lines of defense against risks from AI. 2022. arXiv: 2212.08364 [cs.CY]. · arXiv
  90. Peter Cihon et al. "AI Certification: Advancing Ethical Practice by Reducing Information Asym- metries". In: IEEE Transactions on Technology and Society 2.4 (Dec. 2021), pp. 200-209. DOI: 10.1109/tts.2021.3077595. · DOI
  91. Philip F.S. Berg. "Unfit to Serve: Permanently Barring People from Serving as Officers and Directors of Publicly Traded Companies After theOfficers and Directors of Publicly Traded Companies After the Sarbanes-Oxley ActSarbanes-Oxley Act". In: Vanderbilt Law ReviewVanderbilt Law Review 56.6 ().
  92. Gary S. Becker. "A Theory of Competition among Pressure Groups for Political Influence". In: The Quarterly Journal of Economics 98 (1983), pp. 371-395.
  93. Daniel Carpenter and David Moss, eds. Preventing Regulatory Capture: Special Interest Influence and How to Limit It. Cambridge University Press, 2013.
  94. Leon Derczynski et al. Assessing Language Model Deployment with Risk Cards. 2023. arXiv: 2303.18190 [cs.CL]. · arXiv
  95. Percy Liang et al. Holistic Evaluation of Language Models. 2022. arXiv: 2211.09110 [cs.CL]. · arXiv
  96. Stella Biderman et al. Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling. 2023. arXiv: 2304.01373 [cs.CL]. · arXiv
  97. Aarohi Srivastava et al. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. 2022. arXiv: 2206.04615 [cs.CL]. · arXiv
  98. Dan Hendrycks et al. Measuring Massive Multitask Language Understanding. 2021. arXiv: 2009. 03300 [cs.CY].
  99. Deep Ganguli et al. Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned. 2022. arXiv: 2209.07858 [cs.CL]. · arXiv
  100. Ethan Perez et al. Red Teaming Language Models with Language Models. 2022. arXiv: 2202.03286 [cs.CL]. · arXiv
  101. Miles Brundage et al. Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. 2020. arXiv: 2004.07213 [cs.CY]. · arXiv
  102. Leo Gao, John Schulman, and Jacob Hilton. Scaling Laws for Reward Model Overoptimization. 2022. arXiv: 2210.10760 [cs.LG]. · arXiv
  103. Samuel R. Bowman et al. Measuring Progress on Scalable Oversight for Large Language Models. 2022. arXiv: 2211.03540 [cs.HC]. · arXiv
  104. Samir Passi and Mihaela Vorvoreanu. Overreliance on AI: Literature Review. AI Ethics, Effects in Engineering, and Research. June 2022.
  105. Ziwei Ji et al. "Survey of Hallucination in Natural Language Generation". In: ACM Computing Surveys 55.12 (Mar. 2023), pp. 1-38. DOI: 10.1145/3571730. · DOI · Paper page
  106. Samuel Gehman et al. "RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models". In: Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, 2020. DOI: 10.18653/v1/2020.findings-emnlp.301. · DOI · Paper page
  107. Amanda Askell et al. A General Language Assistant as a Laboratory for Alignment. 2021. arXiv: 2112.00861 [cs.CL]. · arXiv
  108. Ian Tenney et al. The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models. 2020. arXiv: 2008.05122 [cs.CL]. · arXiv
  109. Shoaib Ahmed Siddiqui et al. Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics. 2022. arXiv: 2209.10015 [cs.LG]. · arXiv
  110. Toby Shevlane and Allan Dafoe. The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse? 2020. arXiv: 2001.00463 [cs.CY]. · arXiv
  111. OECD. OECD Framework for the Classification of AI systems. Feb. 2022. DOI: 10.1787/cb6d9eca- en. · DOI
  112. Irene Solaiman et al. Evaluating the Social Impact of Generative AI Systems in Systems and Society. 2023. arXiv: 2306.05949 [cs.CY]. · arXiv
  113. Geoffrey Irving, Paul Christiano, and Dario Amodei. AI safety via debate. 2018. arXiv: 1805.00899 [stat.ML]. · arXiv
  114. Elisabeth Keller and Gregory A. Gehlmann. "Introductory comment: a historical introduction to the Securities Act of 1933 and the Securities Exchange Act of 1934". In: Ohio State Law Journal 49 (1988), pp. 329-352.
  115. Inioluwa Deborah Raji and Joy Buolamwini. "Actionable Auditing". In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. ACM, Jan. 2019. DOI: 10 . 1145 / 3306618 . 3314244. · DOI · Paper page
  116. Jakob Mökander et al. "Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations". In: Science and Engineering Ethics 27.4 (July 2021). DOI: 10.1007/ s11948-021-00319-4. · DOI
  117. Gregory Falco et al. "Governing AI safety through independent audits". In: Nature Machine Intelli- gence 3.7 (July 2021), pp. 566-571. DOI: 10.1038/s42256-021-00370-7. · DOI · Paper page
  118. Inioluwa Deborah Raji et al. "Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance". In: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. ACM, July 2022. DOI: 10.1145/3514094.3534181. · DOI · Paper page
  119. Sasha Costanza-Chock, Inioluwa Deborah Raji, and Joy Buolamwini. "Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem". In: 2022 ACM Conference on Fairness, Accountability, and Transparency. ACM, June 2022. DOI: 10.1145/3531146.3533213. · DOI · Paper page
  120. Daniel M. Ziegler et al. Fine-Tuning Language Models from Human Preferences. 2020. arXiv: 1909.08593 [cs.CL]. · arXiv
  121. Jesse Dodge et al. Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping. 2020. arXiv: 2002.06305 [cs.CL]. · arXiv
  122. Pengfei Liu et al. Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. 2021. arXiv: 2107.13586 [cs.CL]. · arXiv
  123. Xiang Lisa Li and Percy Liang. Prefix-Tuning: Optimizing Continuous Prompts for Generation. 2021. arXiv: 2101.00190 [cs.CL]. · arXiv
  124. Eric Wallace et al. Universal Adversarial Triggers for Attacking and Analyzing NLP. 2021. arXiv: 1908.07125 [cs.CL]. · arXiv
  125. Jonas Schuett. AGI labs need an internal audit function. 2023. arXiv: 2305.17038 [cs.CY]. · arXiv
  126. Richard Worthington. "The Social Control of Technology". In: American Political Science Review 76.1 (Mar. 1982), pp. 134-135. DOI: 10.2307/1960465. · DOI
  127. Competition and Markets Authority. CMA launches initial review of artificial intelligence models. May 4, 2023. URL: https : / / www . gov . uk / government / news / cma -launches -initial - review-of-artificial-intelligence-models.
  128. Jonas Schuett. "Defining the scope of AI regulations". In: Law, Innovation and Technology 15.1 (Jan. 2023), pp. 60-82. DOI: 10.1080/17579961.2023.2184135. · DOI
  129. Robert Baldwin, Martin Cave, and Martin Lodge. Understanding Regulation. Theory, Strategy, and Practice. Oxford: Oxford University Press, 2011. 568 pp. ISBN: 9780199576098.
  130. Louis Kaplow. "Rules versus Standards: An Economic Analysis". In: Duke Law Journal 42.3 (Dec. 1992), pp. 557-629.
  131. Danny Hernandez and Tom B. Brown. Measuring the Algorithmic Efficiency of Neural Networks. 2020. arXiv: 2005.04305 [cs.LG]. · arXiv
  132. Jaime Sevilla et al. Compute Trends Across Three Eras of Machine Learning. 2022. arXiv: 2202. 05924 [cs.LG].
  133. Ben Sorscher et al. Beyond neural scaling laws: beating power law scaling via data pruning. 2023. arXiv: 2206.14486 [cs.LG]. · arXiv
  134. Deep Ganguli et al. "Predictability and Surprise in Large Generative Models". In: 2022 ACM Conference on Fairness, Accountability, and Transparency. ACM, June 2022. DOI: 10 . 1145 / 3531146.3533229. URL: https://doi.org/10.1145%5C%2F3531146.3533229. · DOI · Paper page

Cited by

Sorted by citations

Authors
Anderljung, M., Barnhart, J., Korinek, A., Leung, J., O'Keefe, C., Whittlestone, J., Avin, S., Brundage, M., Bullock, J., Cass-Beggs, D., Chang, B., Collins, T., Fist, T., Hadfield, G., Hayes, A., Ho, L., Hooker, S., Horvitz, E., Kolt, N., Schuett, J., Shavit, Y., Siddarth, D., Trager, R., & Wolf, K.
Affiliations
  • Center for a New American Security (US)
  • Google DeepMind (United Kingdom) (GB)
  • Brookings Institution (US)
  • OpenAI (United States) (US)
  • University of Cambridge (GB)
  • University of Washington (US)
  • Centre for International Governance Innovation (CA)
  • Harvard University (US)
  • Coherent (United States) (US)
  • Microsoft (United States) (US)
  • University of Toronto (CA)
  • The Collective Intelligence Project (US)
  • University of California, Los Angeles (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2307.03718
arXiv
2307.03718
OpenAlex
W4383860829
Cited by
77
References
4
Citations by year
2026: 9 · 2025: 21 · 2024: 27 · 2023: 20
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
Keywords
frontier AI, AI regulation, foundation models, public safety, AI risk management, safety standards, AI licensing, post-deployment monitoring, industry self-regulation, regulatory compliance, AI misuse prevention

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review