How to design an AI ethics board
Open accessAbstract
The development and deployment of artificial intelligence (AI) systems poses significant risks to society. To reduce these risks to an acceptable level, AI companies need an effective risk management process and sound risk governance. In this paper, we explore a particular way in which AI companies can improve their risk governance: by setting up an AI ethics board. We identify five key design choices: (1) What responsibilities should the board have? (2) What should its legal structure be? (3) Who should sit on the board? (4) How should it make decisions? (5) And what resources does it need? We break each of these questions down into more specific sub-questions, list options, and discuss how different design choices affect the board’s ability to reduce societal risks from AI. Several failures have shown that designing an AI ethics board can be challenging. This paper provides a toolbox that can help AI companies to overcome these challenges.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The European regulatory model based on risk-based use case scenarios can serve as a guiding light for the maturity and implementation of ethical, legal and technical frameworks, fostering the creation of industrial and institutional instruments (e.g. AI sandboxes or AI ethics board) that guarantee that AI-based products and services comply with their requirements.
“For this to occur, the European regulatory model based on riskbased use case scenarios can serve as a guiding light for the maturity and implementation of ethical, legal and technical frameworks, fostering the creation of industrial and institutional instruments (e.g. AI sandboxes or AI ethics board [149]) that guarantee that AI-based products and services comply with their requirements.”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [149]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · see citing works
Principles alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886 · cited 1× · see citing works
“Many AI companies have ethics principles [69, 81], but “principles alone cannot guarantee ethical AI” [102].”
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Tabassi, E. (2023) · doi:10.6028/nist.ai.100-1 · Cited by 1,136 · see citing works
Morley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00165-5 · Cited by 882 · cited 1× · see citing works
“They are necessarily vague and need to be put into practice [107, 151, 180].”
Translating Principles into Practices of Digital Ethics: Five Risks of Being Unethical
Floridi, L. (2019) · Philosophy & Technology · doi:10.1007/s13347-019-00354-x · Cited by 524 · cited 1× · see citing works
“This practice is called “ethics washing” [25, 151, 171] or “bluewashing” [56] analog to “greenwashing” [50].”
Silver, D., Singh, S., Precup, D., & Sutton, R. S. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103535 · Cited by 445 · see citing works
Compute Trends Across Three Eras of Machine Learning
Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., & Villalobos, P. (2022) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9891914 · Cited by 363 · cited 1× · see citing works
“to pay increasing costs for compute [105]).”
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363 · see citing works
The Emerging Threat of Ai-driven Cyber Attacks: A Review
Guembe, B., Azeta, A., Misra, S., Osamor, V. C., Fernandez-Sanz, L., & Pospelova, V. (2022) · Applied Artificial Intelligence · doi:10.1080/08839514.2022.2037254 · Cited by 267 · see citing works
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243 · cited 1× · see citing works
“For example, systems might become capable of deceiving their operators and only “pretending” to be safe in a testing environment [116, 132].”
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243 · cited 1× · see citing works
“For example, systems might become capable of deceiving their operators and only “pretending” to be safe in a testing environment [116, 132].”
Preventing repeated real world AI failures by cataloging incidents: the AI incident database
McGregor, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17817 · Cited by 156 · see citing works
Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance
Raji, I. D., Xu, P., Honigsberg, C., & Ho, D. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534181 · Cited by 122 · see citing works
The Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90 · see citing works
Maas, M. M. (2019) · Contemporary Security Policy · doi:10.1080/13523260.2019.1576464 · Cited by 86 · cited 1× · see citing works
“if they lead to an equipment of nuclear weapons with AI technology [64]).”
Truth, Lies, and Automation: How Language Models Could Change Disinformation
Center for Security and Emerging Technology, Buchanan, B., Lohn, A., Musser, M., & Sedova, K. (2021) · doi:10.51593/2021ca003 · Cited by 74 · see citing works
AI ethics: from principles to practice
Zhou, J., & Chen, F. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01602-z · Cited by 72 · see citing works
Model evaluation for extreme risks
Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., Kokotajlo, D., Marchal, N., Anderljung, M., Kolt, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.15324 · Cited by 66 · cited 2× · see citing works
“If certain dangerous model capabilities are detected [157], the company may want to contact government [112] and coordinate with other AI companies to pause capabilities research [2].”
Seger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., Winter, C., Arnold, M., Ó hÉigeartaigh, S., Korinek, A., et al. (2023) · doi:10.2139/ssrn.4596436 · Cited by 31 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 79
- Alaga, J., Schuett, J.: Coordinated pausing: An evaluation-based coordination scheme for frontier AI developers. arXiv preprint arXiv: 2310. 00374, 2023.
- Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv preprint arXiv: 1606. 06565, 2016.
- Anderljung, M., Barnhart, J., Korinek, A., Leung, J., O'Keefe, C., Whittlestone, J., Avin, S., Brundage, M., Bullock, J., Cass-Beggs, D., et al.: Frontier AI regulation: Managing emerging risks to public safety. arXiv preprint arXiv: 2307. 03718, 2023.
- Anderljung, M., Hazell, J.: Protecting society from AI misuse: When are restrictions on capabilities warranted? arXiv preprint arXiv: 2303. 09377, 2023.
- Anderljung, M., Smith, E.T., O'Brien, J., Soder, L., Bucknall, B., Bluemke, E., Schuett, J., Trager, R., Strahm, L., Chowdhury, R.: Towards publicly accountable frontier LLMs: Building an external scrutiny ecosystem under the ASPIRE framework. arXiv preprint arXiv: 2311. 14711, 2023.
- Apaza, C.R., Chang, Y.: What makes whistleblowing effective: Whistleblowing in Peru and South Korea. Public Integrity 13(2), 113-130 (2011). https:// doi. org/ 10. 2753/ PIN10 99-99221 30202 · DOI
- Armstrong, S., Bostrom, N., Shulman, C.: Racing to the preci- pice: A model of artificial intelligence development. AI & Soc. 31, 201-206 (2016). https:// doi. org/ 10. 1007/ s00146-015-0590-y · DOI
- Arnold, Z., Toner, H.: AI accidents: An emerging threat. Center for Security and Emerging Technology, Georgetown Univer- sity (2021). https:// doi. org/ 10. 51593/ 20200 072 · DOI
- van Asselt, M.B., Renn, O.: Risk governance. J. Risk Res. 14(4), 431-449 (2011). https:// doi. org/ 10. 1080/ 13669 877. 2011. 553730 · DOI · Paper page
- Aven, T.: On some recent definitions and analysis frameworks for risk, vulnerability, and resilience. Risk Anal. 31(4), 515-522 (2011). https:// doi. org/ 10. 1111/j. 1539-6924. 2010. 01528.x · DOI
- Aven, T.: On the meaning of a black swan in a risk context. Saf. Sci. 57, 44-51 (2013). https:// doi. org/ 10. 1016/j. ssci. 2013. 01. 016 · DOI
- Baldwin, R., Black, J.: Driving priorities in risk-based regulation: What's the problem? J. Law Soc. 43(4), 565-595 (2016). https:// doi. org/ 10. 1111/ jols. 12003 · DOI
- Belfield, H.: Activism by the AI community: Analysing recent achievements and future prospects. In Proceedings of the AAAI/ ACM Conference on AI, Ethics, and Society, pp. 15-21 (2020). https:// doi. org/ 10. 1145/ 33756 27. 33758 14 · DOI
- Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Harari, Y.N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., Hadfield, G., et al.: Managing AI risks in an era of rapid progress. arXiv pre- print arXiv: 2310. 17688, 2023.
- Bernstein, M.S., Levi, M., Magnus, D., Rajala, B.A., Satz, D., Waeiss, Q.: Ethics and society review: Ethics reflection as a precondition to research funding. PNAS 118(52), e2117261118 (2021). https:// doi. org/ 10. 1073/ pnas. 21172 61118 · DOI
- Bietti, E.: From ethics washing to ethics bashing: A view on tech ethics from within moral philosophy. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 210-219 (2020). https:// doi. org/ 10. 1145/ 33510 95. 33728 60 · DOI · Paper page
- Birhane, A., Isaac, W., Prabhakaran, V., Díaz, M., Elish, M.C., Gabriel, I., Mohamed, S.: Power to the people? Opportunities and challenges for participatory AI. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mecha- nisms, and Optimization, pp. 1-8 (2022). https:// doi. org/ 10. 1145/ 35516 24. 35552 90 · DOI · Paper page
- Birkett, B.S.: The recent history of corporate audit committees. Accounting Historians Journal 13(2), 109-124 (1986).
- Bjørkelo, B.: Workplace bullying after whistleblowing: future research and implications. J. Manag. Psychol. 28(3), 306-323 (2013). https:// doi. org/ 10. 1108/ 02683 94131 13211 78 · DOI
- Bostrom, N.: Existential risks: Analyzing human extinction sce- narios and related hazards. Journal Evol Technol, 9(1), 2001.
- Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Gar- finkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al.: The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. arXiv preprint arXiv: 1802. 07228, 2018.
- Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Had- field, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., et al.: Toward trustworthy AI development: Mechanisms for supporting verifi- able claims. arXiv preprint arXiv: 2004. 07213, 2020.
- Buiten, M.: Towards intelligent regulation of artificial intelli- gence. Eur J Risk Regul 10(1), 41-59 (2019)
- Carlsmith, J.: Is power-seeking AI an existential risk? arXiv pre- print arXiv: 2206. 13353, 2022.
- Cave, S., ÓhÉigeartaigh, S.: An AI race for strategic advan- tage: Rhetoric and risks. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 36-40 (2018). https:// doi. org/ 10. 1145/ 32787 21. 32787 80 · DOI
- Chamberlain, J.: The risk-based approach of the European Union's proposed artificial intelligence regulation: Some com- ments from a tort law perspective. Eur. J. Risk Regul. 14(1), 1-13 (2022). https:// doi. org/ 10. 1017/ err. 2022. 38 · DOI
- Christiano, P., Leike, J., Brown, T.B., Martic, M., Legg, S., Amo- dei, D.: Deep reinforcement learning from human preferences. arXiv preprint arXiv: 1706. 03741, 2017.
- Cihon, P., Schuett, J., Baum, S.D.: Corporate governance of arti- ficial intelligence in the public interest. Information 12(7), 275 (2021). https:// doi. org/ 10. 3390/ info1 20702 75 · DOI
- Cotton-Barratt, O., Daniel, M., Sandberg, A.: Defence in depth against human extinction: Prevention, response, resilience, and why they all matter. Global Pol. 11(3), 271-282 (2020). https:// doi. org/ 10. 1111/ 1758-5899. 12786 · DOI
- Crofts, P., van Rijswijk, H.: Negotiating "evil": Google, Project Maven and the corporate form. Law Technol. Hum. 2(1), 1-16 (2020). https:// doi. org/ 10. 5204/ lthj. v2i1. 1313 · DOI
- Davies, H., Zhivitskaya, M.: Three lines of defence: A robust organising framework, or just lines in the sand? Global Pol. 9, 34-42 (2018). https:// doi. org/ 10. 1111/ 1758-5899. 12568 · DOI
- de Freitas, S.V., Sobral, M.F.F., Ribeiro, A.R.B., da Luz Soare, G.R.: Concepts and forms of greenwashing: A systematic review. Environ. Sci. Eur. 32, 19 (2020). https:// doi. org/ 10. 1186/ s12302-020-0300-3 · DOI · Paper page
- Douek, E.: The Meta Oversight Board and the empty promise of legitimacy. Harv. J. Law Technol. 37 (forthcoming). https:// ssrn. com/ abstr act= 45651 80
- Dungan, J., Waytz, A., Young, L.: The psychology of whistle- blowing. Curr. Opin. Psychol. 6, 129-133 (2015). https:// doi. org/ 10. 1016/j. copsyc. 2015. 07. 005 · DOI
- Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., et al.: Governing AI safety through independent audits. Nat. Mach. Intell. 3(7), 566-571 (2021). https:// doi. org/ 10. 1038/ s42256-021-00370-7 · DOI · Paper page
- Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kada- vath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al.: Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv: 2209. 07858, 2022.
- Goldstein, J.A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., Sedova, K.: Generative language models and automated influ- ence operations: Emerging threats and potential mitigations. arXiv preprint arXiv: 2301. 04246, 2023.
- Grimes, M.G., Williams, T.A., Zhao, E.Y.: Anchors aweigh: The sources, variety, and challenges of mission drift. Acad. Manag. Rev. 44(4), 819-845 (2019). https:// doi. org/ 10. 5465/ amr. 2017. 0254 · DOI
- Gupta, A., Heath, V.: AI ethics groups are repeating one of soci- ety's classic mistakes. MIT Technology Review. https:// www. techn ology review. com/ 2020/ 09/ 14/ 10083 23/ ai-ethics-repre senta tion-artifi cial-intel ligen ce-opini on (2020). Accessed 8 Jan 2024
- Harris, D.: Deepfakes: False pornography is here and the law cannot protect you. Duke L. & Tech. Rev. 17(1), 99-128 (2018).
- Hazell, J.: Large language models can be used to effectively scale spear phishing campaigns. arXiv preprint arXiv: 2305. 06972, 2023.
- Hendrycks, D., Carlini, N., Schulman, J., Steinhardt, J.: Unsolved problems in ML safety. arXiv preprint arXiv: 2109. 13916, 2022.
- Hendrycks, D., Mazeika, M., Woodside, T.: An overview of cata- strophic AI risks. arXiv preprint arXiv: 2306. 12001, 2023.
- Ho, L., Barnhart, J., Trager, R., Bengio, Y., Brundage, M., Car- negie, A., Chowdhury, R., Dafoe, A., Hadfield, G., Levi, M., Snidal, D.: International institutions for advanced AI. arXiv pre- print arXiv: 2307. 04699, 2023.
- Jordan, S.R.: Designing artificial intelligence review boards: Creating risk metrics for review of AI. In IEEE International Symposium on Technology and Society, pp. 1-7 (2019). https:// doi. org/ 10. 1109/ ISTAS 48451. 2019. 89379 42 · DOI
- Jubb, P.B.: Whistleblowing: a restrictive definition and inter- pretation. J. Bus. Ethics 21, 77-94 (1999). https:// doi. org/ 10. 1023/A: 10059 22701 763 · DOI
- Klonick, K.: Insight the making of Facebook's supreme court. The New Yorker. https:// www. newyo rker. com/ tech/ annals-of- techn ology/ inside-the-making-of-faceb ooks-supre me-court (2021). Accessed 8 Jan 2024
- Kolev, K.D., Wangrow, D.B., Barker, V.L., III., Schepker, D.J.: Board committees in corporate governance: A cross-discipli- nary review and agenda for the future. J. Manage. Stud. 56(6), 1138-1193 (2019). https:// doi. org/ 10. 1111/ joms. 12444 · DOI
- Lalley, S.P., Weyl, E.G.: Quadratic voting: How mechanism design can radicalize democracy. AEA Papers and Proceedings 108, 33-37 (2018). https:// doi. org/ 10. 1257/ pandp. 20181 002 · DOI
- Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasu- naga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al.: Holistic evaluation of language models. arXiv preprint arXiv: 2211. 09110, 2022.
- Lundqvist, S.A.: Why firms implement risk governance: stepping beyond traditional risk management to enterprise risk management. J. Account. Public Policy 34(5), 441-466 (2015). https:// doi. org/ 10. 1016/j. jaccp ubpol. 2015. 05. 002 · DOI
- Maas, M.M.: Aligning AI regulation to sociotechnical change. In The Oxford Handbook of AI Governance (2022). https:// doi. org/ 10. 1093/ oxfor dhb/ 97801 97579 329. 013. 22 · DOI
- Mahler, T.: Between risk management and proportionality: The risk-based approach in the EU's Artificial Intelligence Act proposal. In Nordic Yearbook of Law and Informatics, pp. 247-270 (2021). https:// doi. org/ 10. 53292/ 208f5 901. 38a67 238 · DOI
- Mazri, C.: (Re) defining emerging risks. Risk Anal. 37(11), 2053-2065 (2017). https:// doi. org/ 10. 1111/ risa. 12759 · DOI
- Mohamed, S., Png, M.-T., Isaac, W.: Decolonial AI: Decolo- nial theory as sociotechnical foresight in artificial intelligence. Phil. & Technol. 33, 659-684 (2020). https:// doi. org/ 10. 1007/ s13347-020-00405-8 104. · DOI · Paper page
- Mökander, J., Floridi, L.: Operationalising AI governance through ethics-based auditing: An industry case study. AI Ethics 3, 451-468 (2022). https:// doi. org/ 10. 1007/ s43681-022-00171-7 105. · DOI
- Mökander, J., Morley, J., Taddeo, M., Floridi, L.: Ethics-based auditing of automated decision-making systems: Nature, scope, and limitations. Sci. Eng. Ethics 27(44), 1-30 (2021). https:// doi. org/ 10. 1007/ s11948-021-00319-4 106. · DOI
- Mökander, J., Schuett, J., Kirk, H.R., Floridi, L.: Auditing large language models: A three-layered approach. AI Ethics, 1-31 (2023). https:// doi. org/ 10. 1007/ s43681-023-00289-2 107. · DOI · Paper page
- Morley, J., Elhalal, A., Garcia, F., Kinsey, L., Mökander, J., Floridi, L.: Ethics as a service: A pragmatic operationalisation of AI ethics. Mind. Mach. 31(2), 239-256 (2021). https:// doi. org/ 10. 1007/ s11023-021-09563-w 109. · DOI
- Morris, M.R., Sohl-dickstein, J., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C., Legg, S.: Levels of AGI: Operational- izing progress on the path to AGI. arXiv preprint arXiv: 2311. 02462, 2023. 111. Mouton, C.A., Lucas, C., Guest, E.: The operational risks of AI in large-scale biological attacks: A red-team approach. RAND Corporation (2023). https:// doi. org/ 10. 7249/ RRA29 77-1 112. · DOI
- Naudé, W., Dimitri, N.: The race for an artificial general intel- ligence: Implications for public policy. AI & Soc. 35, 367-379 (2020). https:// doi. org/ 10. 1007/ s00146-019-00887-x 114. · DOI
- Near, J.P., Miceli, M.P.: Effective whistle-blowing. Acad. Manag. Rev. 20(3), 679-708 (1995). https:// doi. org/ 10. 2307/ 258791 115. · DOI
- Posner, E.A., Weyl, E.G.: Quadratic voting as efficient corporate governance. U. Chi. L. Rev. 81(1), 251-272 (2014). 138. Raji, I.D., Buolamwini, J.: Actionable auditing: Investigat- ing the impact of publicly naming biased performance results of commercial AI products. In Proceedings of the 2019 AAAI/ ACM Conference on AI, Ethics, and Society, pp. 429-435 (2019). https:// doi. org/ 10. 1145/ 33066 18. 33142 44 139 · DOI · Paper page
- Rando, J., Paleka, D., Lindner, D., Heim, L., Tramèr, F.: Red- teaming the Stable Diffusion safety filter. arXiv preprint arXiv: 2210. 04610, 2022. 141. Sætra, H.S.: A framework for evaluating and disclosing the ESG related impacts of AI with the SDGs. Sustainability 13(15), 8503 (2021). https:// doi. org/ 10. 3390/ su131 58503 142. · DOI
- Sandbrink, J.B., Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools. arXiv preprint arXiv: 2306. 13952, 2023. 143.
- Sandbrink, J.B., Hobbs, H., Swett, J., Dafoe, A., Sandberg, A.: Differential technology development: A responsible innovation principle for navigating technology risks. SSRN. https:// ssrn. com/ abstr act= 42136 70 (2022). Accessed 8 Jan 2024 144.
- Schuett, J.: AGI labs need an internal audit function. arXiv pre- print arXiv: 2305. 17038, 2023. 147. Schuett, J.: Three lines of defense against risks from AI. AI & Soc. (2023). https:// doi. org/ 10. 1007/ s00146-023-01811-0 148. · DOI · Paper page
- Schuett, J.: Risk management in the Artificial Intelligence Act. Eur. J. Risk Regul., 1-19 (2023). https:// doi. org/ 10. 1017/ err. 2023.1 149. Schuett, J., Dreksler, N., Anderljung, M., McCaffary, D., Heim, L., Bluemke, E., Garfinkel, B.: Towards best practices in AGI safety and governance: A survey of expert opinion. arXiv pre- print arXiv: 2305. 07153, 2023. 150. Seele, P., Schultz, M.D.: From greenwashing to machine wash- ing: A model and future directions derived from reasoning by analogy. J. Bus. Ethics 178, 1063-1089 (2022). https:// doi. org/ 10. 1007/ s10551-022-05054-9 151. · DOI
- Seger, E.: In defence of principlism in AI ethics and govern- ance. Philos. Technol. 35(45), 1-7 (2022). https:// doi. org/ 10. 1007/ s13347-022-00538-y 152. · DOI
- Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla, N., Gallegos, J., Smart, A., Garcia, E., Virk, G.: Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction. arXiv preprint arXiv: 2210. 05791, 2022. 155. Shevlane, T.: Structured access: An emerging paradigm for safe AI deployment. In The Oxford Handbook of AI Governance (2022). https:// doi. org/ 10. 1093/ oxfor dhb/ 97801 97579 329. 013. 39 156 · DOI
- Shevlane, T., Dafoe, A.: The offense-defense balance of scien- tific knowledge: Does publishing AI research reduce misuse? In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 173-179 (2020). https:// doi. org/ 10. 1145/ 33756 27. 33758 15 157 · DOI · Paper page
- Singer, P., Tse, Y.F.: AI ethics: The case for including ani- mals. AI Ethics 3, 539-551 (2023). https:// doi. org/ 10. 1007/ s43681-022-00187-z 160. · DOI
- Trager, R., Harack, B., Reuel, A., Carnegie, A., Heim, L., Ho, L., Kreps, S., Lall, R., Larter, O., ÓhÉigeartaigh, S., et al.: Inter- national governance of civilian AI: A jurisdictional certification approach. arXiv preprint arXiv: 2308. 15514, 2023. 168. UK Department for Science, Innovation and Technology: Emerg- ing processes for frontier AI safety. https:// www. gov. uk/ gover nment/ publi catio ns/ emerg ing-proce sses-for-front ier-ai-safety (2023). Accessed 8 Jan 2024 169.
- Urbina, F., Lentzos, F., Invernizzi, C., Ekins, S.: Dual use of artificial-intelligence-powered drug discovery. Nat. Mach. Intell. 4(3), 189-191 (2022). https:// doi. org/ 10. 1038/ s42256-022-00465-9 170. · DOI · Paper page
- van Duuren, E., Plantinga, A., Scholtens, B.: ESG integration and the investment management process: Fundamental investing reinvented. J. Bus. Ethics 138, 525-533 (2016). https:// doi. org/ 10. 1007/ s10551-015-2610-8 171. · DOI
- van Maanen, G.: AI ethics, ethics washing, and the need to politi- cize data ethics. DISO 1(9), 1-3 (2022). https:// doi. org/ 10. 1007/ s44206-022-00013-3 172. · DOI
- Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al.: Ethical and social risks of harm from language models. arXiv preprint arXiv: 2112. 04359, 2021. 175. Westerlund, M.: The emergence of deepfake technology: a review. Technol. Innov. Manag. Rev. 9(11), 39-52 (2019). 176. Whyte, C.: Deepfake news: AI-enabled disinformation as a multi- level public policy challenge. J. Cyber Policy 5(2), 199-217 (2020). https:// doi. org/ 10. 1080/ 23738 871. 2020. 17971 35 177 · DOI
- Wong, D., Floridi, L.: Meta's Oversight Board: A review and critical assessment. Minds Mach. 33, 261-284 (2022). https:// doi. org/ 10. 1007/ s11023-022-09613-x 178. · DOI
- Ziegler, D.M., Stiennon, N., Wu, J., Brown, T.B., Radford, A., Amodei, D., Christiano, P., Irving, G.: Fine-tuning language models from human preferences. arXiv preprint arXiv: 1909. 08593, 2019.
Cited by
Sorted by citations
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800 · see citing works
Three lines of defense against risks from AI
Schuett, J. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01811-0 · Cited by 41 · see citing works
Self-citation
- Authors
- Schuett, J., Reuel, A.-K., & Carlier, A.
- Affiliations
- Centre for the Governance of AI (GB)
- Corresponding author
- Schuett, Jonas
- Venue
- AI and Ethics
- Published
- 2024-02-15
- Volume
- 5
- Issue
- 2
- Pages
- 863-881
- ISSN
- 2730-5953 (ISSN-L)
- 2730-5961
- OpenAlex
- W4391843890
- Cited by
- 46
- References
- 22
- Citations by year
- 2026: 16 · 2025: 15 · 2024: 11 · 2023: 4
- FWCI
- 17.46
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- AI and Ethics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Decision Sciences › Information Systems and Management › Ethics in Business and Education
- Keywords
- AI governance, risk governance, AI risk management, board design, societal risks of AI, board decision-making, board composition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review