Three lines of defense against risks from AI
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Organizations that develop and deploy artificial intelligence (AI) systems need to manage the associated risks—for economic, legal, and ethical reasons. However, it is not always clear who is responsible for AI risk management. The three lines of defense (3LoD) model, which is considered best practice in many industries, might offer a solution. It is a risk management framework that helps organizations to assign and coordinate risk management roles and responsibilities. In this article, I suggest ways in which AI companies could implement the model. I also discuss how the model could help reduce risks from AI: it could identify and close gaps in risk coverage, increase the effectiveness of risk management practices, and enable the board of directors to oversee management more effectively. The article is intended to inform decision-makers at leading AI companies, regulators, and standard-setting bodies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Internal audits-such as the model evaluations required by the US Federal Reserve's Supervisory Letter (SR) 11-7 [9]-are forms of self-assessment: evaluation conducted by the audited party or by outside contractants without robust safeguards against conflicts of interest [2,57].”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [57]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Research on algorithm auditing has so far called for auditor professionalization and certification as one of the requirements for high-quality audits [18,57].”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is a growing consensus that, with internal audits, conflicts of interest between the auditor and the auditee may undermine their purpose of providing the assurance of trustworthiness or compliance to outside parties [2,23,28,57,66,75].”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is convergence on the importance of external audits for providing assurance to outside parties, such as to governments, the public, or other organizations[2,28,46,57,66].”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While the idea to follow the financial auditing practices-at least in part-is in itself not new [27,44,57], our paper offers an actionable approach for performing external compliance and assurance audits which is missing in the literature.”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · cited 1×
“…bias, i.e. the tendency to “assess the frequency of a class or the probability of an event by the ease with which instances or occurrences can be brought to mind” (Tversky and Kahneman 1974)], and other human errors (e.g. a person filling out a risk register slips a line), among other things.”
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Tabassi, E. (2023) · doi:10.6028/nist.ai.100-1 · Cited by 2,951
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Reports will typically include risk registers and risk matrices (IEC 2019), but they could also involve information about specific models, in the form of (preliminary) model cards (Mitchell et al. 2019), data sheets (Gebru et al. 2021), and system cards (Green et al. 2022).”
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 1×
“Reports will typically include risk registers and risk matrices (IEC 2019), but they could also involve information about specific models, in the form of (preliminary) model cards (Mitchell et al. 2019), data sheets (Gebru et al. 2021), and system cards (Green et al. 2022).”
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 2×
“It does not merely mean an audit that is done internally (Raji et al 2020).”
Box, G. E. P. (1976) · Journal of the American Statistical Association · doi:10.1080/01621459.1976.10480949 · Cited by 2,114
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · cited 1×
“Ethical expectations might stem from AI ethics principles that organizations have adopted on a voluntary basis (Jobin et al. 2019; Hagendorff 2020).”
The Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907 · cited 1×
“Ethical expectations might stem from AI ethics principles that organizations have adopted on a voluntary basis (Jobin et al. 2019; Hagendorff 2020).”
Research works printed but not linked: 20
- Alaga J, Schuett J (2023) Coordinated pausing: an evaluation-based coordination scheme for frontier AI developers. arXiv. http:// arxiv. org/ abs/ 2310. 00374
- Bai Y, Kadavath S, Kundu S, Askell A, Kernion J, Jones A, Kaplan J (2022) Constitutional AI: harmlessness from AI feedback. arXiv. http:// arxiv. org/ abs/ 2212. 08073
- Barrett AM, Hendrycks D, Newman J, Nonnecke B (2022) Actionable guidance for high-consequence AI risk management: towards standards addressing AI catastrophic risks. arXiv. http:// arxiv. org/ abs/ 2206. 08966
- Bengio Y, Hinton G, Yao A, Song D, Abbeel P, Harari YN et al (2023) Managing AI risks in an era of rapid progress. arXiv. http:// arxiv. org/ abs/ 2310. 17688
- Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S et al (2021) On the opportunities and risks of foundation models. arXiv. https:// arxiv. org/ abs/ 2108. 07258
- Boţa-Avram C, Palfi C (2009) Measuring and assessment of inter- nal audit's effectiveness. Ann Faculty Econ Univ Oradea 3(1):784-790
- Buolamwini J, Gebru T (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. In: Proceedings of the 1st Conference on Fairness, Accountability and Transpar- ency, p 77-91. https:// perma. cc/ 976J-AR93
- Cao Y, Li S, Liu Y, Yan Z, Dai Y, Yu PS, Sun L (2023) A compre- hensive survey of AI-generated content (AIGC): a history of generative AI from GAN to ChatGPT. arXiv. http:// arxiv. org/ abs/ 2303. 04226
- Chen M, Tworek J, Jun H, Yuan Q, de Pinto HPO, Kaplan J et al (2021) Evaluating large language models trained on code. arXiv. http:// arxiv. org/ abs/ 2107. 03374
- Koessler L, Schuett J (2023) Risk assessment at AGI companies: a review of popular risk assessment techniques from other safety- critical industries. arXiv. https:// arxiv. org/ abs/ 2307. 08823
- Liang P, Bommasani R, Lee T, Tsipras D, Soylu D, Yasunaga M et al (2022) Holistic evaluation of language models. arXiv. http:// arxiv. org/ abs/ 2211. 09110
- O’Brien J, Ee S, Williams Z (2023) Deployment corrections: an incident response framework for frontier AI models. arXiv. http://arxiv.org/abs/2310.00328 · arXiv
- O'Brien J, Ee S, Williams Z (2023) Deployment corrections: an incident response framework for frontier AI models. arXiv. http:// arxiv. org/ abs/ 2310. 00328 OECD (2023) OECD AI incidents monitor. OECD. https:// oecd. ai/ en/ incid ents OpenAI (2023a) OpenAI's approach to frontier risk. OpenAI. https:// perma. cc/ 9YGS-NZVX OpenAI (2023b) Frontier risk and preparedness. OpenAI. https:// perma. cc/ 5AFJ-JZG4 Ord T (2021) Proposal for a new 'three lines of defence' approach to UK risk management. Future of Humanity Institute, University of Oxford. https:// perma. cc/ VHH9-L36R
- Perez E, Ringer S, Lukošiūtė K, Nguyen K, Chen E, Heiner S et al (2022b) Discovering language model behaviors with model- written evaluations. arXiv. http:// arxiv. org/ abs/ 2212. 09251
- Rando J, Paleka D, Lindner D, Heim L, Tramèr F (2022) Red-teaming the stable diffusion safety filter. arXiv. https:// arxiv. org/ abs/ 2210. 04610
- Rupšys R, Boguslauskas V (2007) Measuring performance of internal auditing: empirical evidence. Eng Econ 55(5):9-15
- Schuett J, Dreksler N, Anderljung M, McCaffary D, Heim L, Bluemke E, Garfinkel B (2023) Towards best practices in AGI safety and governance: a survey of expert opinion. arXiv. http:// arxiv. org/ abs/ 2305. 07153
- Walker D (2009) A review of corporate governance in UK banks and other financial industry entities: final recommendations. https:// perma. cc/ 2K9C-EMME
- Zhivitskaya M (2015) The practice of risk oversight since the global financial crisis: closing the stable door? [Doctoral dissertation, The London School of Economics and Political Science]. LSE Theses Online. https:// perma. cc/ KKA6-QK56
- Ziegler DM, Stiennon N, Wu J, Brown TB, Radford A, Amodei D, Irving G (2019) Fine-tuning language models from human pref- erences. arXiv. http:// arxiv. org/ abs/ 1909. 08593
Cited by
Auditing Large Language Models: A Three-Layered Approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023) · doi:10.2139/ssrn.4361607 · Cited by 292
Self-citationAuditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023) · arXiv · doi:10.48550/arxiv.2302.08500 · Cited by 292
Self-citationAuditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
Self-citationAuditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152
“Recent research suggests that this approach could also help reduce the risks posed by AI systems (Schuett, 2022).”
BackgroundMcIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102
“…into various sectors, ethical considerations and legal compliance have become increasingly crucial, which requires a focus on developing ‘Ethical AI Frameworks’, a new category in our taxonomy reflecting the trend towards responsible AI development in generative AI [214], [215], [15], [216], [217].”
Background
Metadata
- Authors
- Schuett, J.
- Affiliations
- Goethe University Frankfurt (DE)
- Corresponding author
- Schuett, Jonas
- ORCID
- Schuett, Jonas
- Venue
- AI & SOCIETY
- Published
- 2023-11-27
- Volume
- 40
- Issue
- 2
- Pages
- 493-507
- ISSN
- 0951-5666 (ISSN-L)
- 1435-5655
- OpenAlex
- W4389052201
- Cited by
- 41
- References
- 72
- Citations by year
- 2026: 14 · 2025: 8 · 2024: 15 · 2023: 4
- FWCI
- 10.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- AI & Society (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Institute of Standards and Technology
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Business, Management and Accounting › Management Information Systems › Information Technology Governance and Strategy
- Keywords
- three lines of defense model, corporate governance, board oversight, AI regulation, AI ethics, AI companies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Three lines of defense against risks from AI. https://app.cochat.ai/papers/10.1007/s00146-023-01811-0
- The paper
- Schuett, J. (2025). Three lines of defense against risks from AI. AI & SOCIETY. https://doi.org/10.1007/s00146-023-01811-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.