Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulation
Open accessThe DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
The proposed European Artificial Intelligence Act (AIA) is the first attempt to elaborate a general legal framework for AI carried out by any major global economy. As such, the AIA is likely to become a point of reference in the larger discourse on how AI systems can (and should) be regulated. In this article, we describe and discuss the two primary enforcement mechanisms proposed in the AIA: the conformity assessments that providers of high-risk AI systems are expected to conduct, and the post-market monitoring plans that providers must establish to document the performance of high-risk AI systems throughout their lifetimes. We argue that the AIA can be interpreted as a proposal to establish a Europe-wide ecosystem for conducting AI auditing, albeit in other words. Our analysis offers two main contributions. First, by describing the enforcement mechanisms included in the AIA in terminology borrowed from existing literature on AI auditing, we help providers of AI systems understand how they can prove adherence to the requirements set out in the AIA in practice. Second, by examining the AIA from an auditing perspective, we seek to provide transferable lessons from previous research about how to refine further the regulatory approach outlined in the AIA. We conclude by highlighting seven aspects of the AIA where amendments (or simply clarifications) would be helpful. These include, above all, the need to translate vague concepts into verifiable criteria and to strengthen the institutional safeguards concerning conformity assessments based on internal checks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The AIA applies to both fully and partially automated systems and envisions that conformity with its norms may be assessed through third-party audits (see further: (Mökander et al. 2022)).”
From Institutionalised distrust and human oversight of artificial intelligence: towards a democratic design of AI governance under the European Union AI Act (2024) · cites this paper as Mökander et al. 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Auditing has been proposed in a variety of laws and regulations [17,22,23,25,42,52,75], 1 standardized frameworks [47,62], and guidelines for industry best practices [5,77] as a mechanism to identify and mitigate risks of harm in artificial intelligence (AI) 2and to build public trust and promote accountability for AI system developers and deployers.”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [42]
The cited source says what the claim attributes to it.
In the citing paper
“The AI Act has been interpreted as the European ecosystem to conduct AI auditing [122].”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2011) · Cited by 14,317
Nudge: improving decisions about health, wealth, and happiness
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0977 · Cited by 11,572
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
The Audit Society: Rituals of VerificationPreprint
Power, M. (1999) · RePEc: Research Papers in Economics · Cited by 3,647
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 4×
“However, external audits are fundamentally limited by a lack of access to internal processes at the audited organisation (Raji et al., 2020).”
BackgroundInfluentialThe Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
The Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
A Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.8cd550d1 · Cited by 1,631 · cited 1×
“However, central questions— such as according to which metrics AI systems should be evaluated and who should be accountable for different system failures—have remained unanswered (Floridi & Cowls, 2019).”
BackgroundSelf-citationThe Brussels Effect: How the European Union Rules the World
Bradford, A. (2020) · doi:10.1093/oso/9780190088583.001.0001 · Cited by 1,349
On the Morality of Artificial Agents
Floridi, L., & Sanders, J. W. (2004) · Minds and Machines · doi:10.1023/b:mind.0000035461.63578.9d · Cited by 1,218 · cited 1×
“Specifically, the combination of relative autonomy, adaptability, and interactivity underpins both beneficial and problematic uses of AI systems (Dignum, 2017; Floridi & Sanders, 2004; Russell & Norvig, 2015).”
BackgroundSelf-citationUnderstanding Regulation: Theory, Strategy, and PracticePreprint
Baldwin, R., Cave, M. E., & Lodge, M. (1999) · RePEc: Research Papers in Economics · Cited by 1,173
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · cited 1×
“1 3 fairness—like individual fairness, demographic parity, and equality of opportunity—are mutually exclusive (Friedler et al., 2016; Kusner et al., 2017).”
Background‘Hypernudge’: big Data as a mode of regulation by design
Yeung, K. (2017) · Information, Communication & Society · doi:10.1080/1369118x.2016.1186713 · Cited by 868 · cited 1×
“However, digital mediation inevitably influences human users, for example, by nudging an individual’s preferences through positive reinforcement or indirect suggestion (Thaler & Sunstein, 2008; Yeung, 2017).”
BackgroundThe Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.07228 · Cited by 849
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.22520 · Cited by 849
How AI can be a force for good
Taddeo, M., & Floridi, L. (2018) · Science · doi:10.1126/science.aat5991 · Cited by 760
Self-citationOn Defining Artificial Intelligence
Wang, P. (2019) · Journal of Artificial General Intelligence · doi:10.2478/jagi-2019-0002 · Cited by 740
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0279-x · Cited by 709 · cited 1×
“And, since they also make mistakes and produce discriminatory or inconsistent outcomes (Kahneman, 2011), the use of AI systems can sometimes lead to more objective and potentially fairer decisions (Lepri et al., 2018).”
BackgroundTranslating Principles into Practices of Digital Ethics: Five Risks of Being Unethical
Floridi, L. (2019) · Philosophy & Technology · doi:10.1007/s13347-019-00354-x · Cited by 524
Self-citationAequitas: A Bias and Fairness Audit Toolkit
Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., & Ghani, R. (2018) · arXiv · doi:10.48550/arxiv.1811.05577 · Cited by 496
The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions
Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314289 · Cited by 399
The Method of Levels of Abstraction
Floridi, L. (2008) · Minds and Machines · doi:10.1007/s11023-008-9113-7 · Cited by 349
Self-citationSoft Ethics and the Governance of the Digital
Floridi, L. (2018) · Philosophy & Technology · doi:10.1007/s13347-018-0303-9 · Cited by 325
Self-citationThe IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems
Chatila, R., & Havens, J. C. (2019) · Intelligent Systems, Control and Automation: Science and Engineering · doi:10.1007/978-3-030-12524-0_2 · Cited by 307
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.07213 · Cited by 306 · cited 1×
“Because external audits help address concerns about the incentives for accuracy in self-reporting (Brundage et al., 2020), they are typically required for formal verification and certification procedures (Yanisky-Ravid & Hallisey, 2018).”
BackgroundTowards a Code of Ethics for Artificial Intelligence
Boddington, P. (2017) · Artificial Intelligence: Foundations, Theory, and Algorithms · doi:10.1007/978-3-319-60648-4 · Cited by 294 · cited 1×
“Hence, any set of ethical principles will only be as good as the public institution backing it (Boddington, 2017).”
BackgroundEthics-Based Auditing to Develop Trustworthy AI
Mökander, J., & Floridi, L. (2021) · Minds and Machines · doi:10.1007/s11023-021-09557-8 · Cited by 268 · cited 1×
“…to consumers that their products are healthy through detailed nutritional labels (Holland et al., 2018), the documentation and communication of the steps taken to ensure that AI systems are ethicallysound can play a positive role in both marketing and public relations (Mökander & Floridi, 2021).”
BackgroundSelf-citationDetecting racial bias in algorithms and machine learning
Turner Lee, N. (2018) · Journal of Information, Communication and Ethics in Society · doi:10.1108/jices-06-2018-0056 · Cited by 260
On the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247
Towards Intelligent Regulation of Artificial Intelligence
BUITEN, M. C. (2019) · European Journal of Risk Regulation · doi:10.1017/err.2019.8 · Cited by 237
Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations
Mökander, J., Morley, J., Taddeo, M., & Floridi, L. (2021) · Science and Engineering Ethics · doi:10.1007/s11948-021-00319-4 · Cited by 207
Self-citationThe algorithm audit: Scoring the algorithms that score us
Brown, S., Davidovic, J., & Hasan, A. (2021) · Big Data & Society · doi:10.1177/2053951720983865 · Cited by 204
Defining the Scope of AI Regulations
Schuett, J. (2019) · Law Innovation and Technology · doi:10.1080/17579961.2023.2184135 · Cited by 203
Bad machines corrupt good morals
Köbis, N., Bonnefon, J.-F., & Rahwan, I. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01128-2 · Cited by 202
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
Holland, S., Hosny, A., Newman, S., Joseph, J., & Chmielinski, K. (2018) · arXiv · doi:10.48550/arxiv.1805.03677 · Cited by 175
The European legislation on AI: A brief analysis of its philosophical approach
Floridi, L. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00460-9 · Cited by 169 · cited 2×
“Throughout this article, we will return to highlight some of the points raised by different commentators (for a brief overview, see Floridi, 2021).”
BackgroundSelf-citationCOVID-19 contact tracing apps: a stress test for privacy, the GDPR, and data protection regimes
Bradford, L., Aboy, M., & Liddell, K. (2020) · Journal of Law and the Biosciences · doi:10.1093/jlb/lsaa034 · Cited by 165
Regulating Artificial Intelligence: Proposal for a Global Solution
Erdélyi, O. J., & Goldsmith, J. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278731 · Cited by 133
Law and Corporate Behaviour: Integrating Theories of Regulation, Enforcement, Compliance and Ethics
Hodges, C. (2015) · Hart Publishing eBooks · doi:10.5040/9781474201124 · Cited by 131
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable ClaimsPreprint
Brundage, M., Avin, S., Wang, J., et al. (2020) · arXiv (Cornell University) · Cited by 124
A governance framework for algorithmic accountability and transparency
Koene, A. R., Clifton, C., Hatada, Y., Webb, H., & Richardson, R. (2019) · Repository@Nottingham (University of Nottingham) · doi:10.2861/59990 · Cited by 104
Dignum, V. (2017) · arXiv · doi:10.48550/arxiv.1706.02513 · Cited by 103
Etzioni, A., & Etzioni, O. (2016) · Ethics and Information Technology · doi:10.1007/s10676-016-9400-6 · Cited by 84 · cited 1×
“…clear—and even AI systems used for internal purposes 40 One option would be to include elements of observation-based impact auditing in the post-market monitoring plan, for example with the help of ‘oversight programs’ (Etzioni & Etzioni, 2016) that monitor and evaluate system outputs continuously.”
BackgroundDignum, V. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/655 · Cited by 84
Transparency as design publicity: explaining and justifying inscrutable algorithms
Loi, M., Ferrario, A., & Viganò, E. (2021) · Ethics and Information Technology · doi:10.1007/s10676-020-09564-w · Cited by 79
Chamon, M. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198784487.001.0001 · Cited by 67
Mökander, J., & Axente, M. (2023) · AI & SOCIETY · doi:10.1007/s00146-021-01286-x · Cited by 62
Self-citationDennehy, D., Griva, A., Pouloudi, N., Dwivedi, Y., Pappas, I. O., & Mäntymäki, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85447-8 · Cited by 35
Defining the Scope of AI RegulationsPreprint
Schuett, J. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3453632 · Cited by 29
What costs should we expect from the EU’s AI Act?Preprint
Haataja, M., & Bryson, J. J. (2021) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/8nzb4 · Cited by 23
Towards a Framework for Ethical Audits of AI Algorithms.
LaBrie, R. C., & Steinke, G. (2019) · Journal of the Association for Information Systems · Cited by 20
Business Ethics - A Philosophical and Behavioral Approach
Conrad, C. A. (2018) · Springer eBooks · doi:10.1007/978-3-319-91575-3 · Cited by 17
Yanisky-Ravid, S., & Hallisey, S. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3278490 · Cited by 14
The European Legislation on AI: A Brief Analysis of its Philosophical ApproachPreprint
Floridi, L. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3873273 · Cited by 10
Self-citationUna defensa del construccionismo: la filosofía como ingeniería conceptual
Floridi, L. (2017) · Pensamiento Revista de investigación e información filosófica · doi:10.14422/pen.v73.i276.y2017.003 · Cited by 8
Self-citationTheory and applications of artificial immune systems
Gao, X.-Z., Chow, M., PELTA, D. A., & Timmis, J. (2010) · Neural Computing and Applications · doi:10.1007/s00521-010-0388-2 · Cited by 7
Placement and evaluation of Southeast Asian children
(1977) · American Journal of Psychiatry · doi:10.1176/ajp.134.8.aj1348938 · Cited by 5
Susi, M. (2019) · Edward Elgar Publishing eBooks · doi:10.4337/9781788971775.00015 · Cited by 2
Research works printed but not linked: 21
- Barrett, C. (2019). Are the EU GDPR and the California CCPA becoming the de facto global standards for data privacy and protection? The SciTech Lawyer, 15(3), 24–29.
- Bradford, A. (2012). The Brussels effect. Law Review, 107(1), 1–68.
- CDEI. (2021a). The European Commission’s artificial intelligence act highlights the need for an effective AI assurance ecosystem. CDEI.
- CDEI. (2021b). The need for effective AI assurance. CDEI.
- ECP. (2018). Artificial intelligence impact assessment. ECP.
- European Commission. (2019). Policy and investment recommendations for trustworthy Artificial Intelligence. European Commission.
- European Commission. (2020a). EU regulation on European data governance 0340. European Commission.
- European Commission. (2020b). Recommendation (EU) 2020/518 of 8 April 2020 on a common union toolbox for the use of technology and data to combat and exit from the COVID-19 crisis, in particular concerning mobile applications and the use of anonymes data, OJ [2020] L 114/7. European Commission.
- European Commission. (2020c). White paper on artificial intelligence—A European approach to excellence and trust (p. 27). European Commission.
- European Commission. (2021a). ANNEXES to the proposal for a regulation of the European Parliament and of the council. European Commission.
- European Commission. (2021b). Commission staff working document: Impact assessment accompanying the proposal for a regulation of the European Parliament and of the Council (Artificial Intelligence ACT). European Commission.
- European Commission. (2021c). Proposal for regulation of the European parliament and of the council—Laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain Union legislative acts. European Commission.
- European Parliament. (2021). Resolution on the Commission evaluation report on the implementation of the general data protection regulation two years after its application. European Commission.
- European Parliament/Council. (2001). Directive 2001/95/EC of the European Parliament and of the Council on general product safety. Official Journal of the European Communities, 4(7), 4–17.
- Fontanelli, F. (2016). The Court of Justice of the European Union and the illusion of balancing in internet related disputes. In O. Pollicino & G. Romeo (Eds.), The internet and constitutional law. The protection of fundamental rights and constitutional adjudication in Europe (pp. 94–117). Routledge.
- ICO. (2020). Guidance on the AI auditing framework: Draft guidance for consultation. Information Commissioner’s Office.
- McCarthy, J. (2007). What is artificial intelligence? Stanford University.
- Mitchell, T. M. (1997). Machine learning. McGraw-Hill.
- O’Donoghue, C., Splittgerber, A., & O’Brien, S. (2021). The proposed European regulation on artificial intelligence—A summary of the obligations, scope and effect. Reed Smith Client Alerts.
- Russell, S. J., & Norvig, P. (2015). Artificial intelligence: A modern approach (3rd ed.). Prentice Hall.
- The European Parliament. (2016). EU general data protection regulation. Official Journal of the European Union, 27 April.
Cited by
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
Laux, J., Wachter, S., & Mittelstadt, B. (2024) · Regulation & Governance · doi:10.1111/rego.12512 · Cited by 365
“This will at least be the case until an external AI auditing infrastructure has emerged, able to certify compliance with the AI Act (see also Section 5.4; Mökander et al., 2022).”
BackgroundInfluentialShaping the future of healthcare: ethical clinical challenges and pathways to trustworthy AI
Goktas, P., & Grzybowski, A. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14051605 · Cited by 336
“…ethical guidelines and governance recommendations provide templates for integrating ethical principles directly into regulatory structures, ensuring that notions of transparency, accountability, fairness, and patient autonomy are not just aspirational ideals but enforceable standards [71,73].”
Auditing Large Language Models: A Three-Layered Approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023) · doi:10.2139/ssrn.4361607 · Cited by 292
Self-citationAuditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“Methodologically, post-deployment monitoring can be done in different ways, e.g., 24This structure mirrors the ‘conformity assessments’ and ‘post-market monitoring plans’ proposed in the EU AI Act [91]. periodically reviewing the output from an application and comparing it to relevant standards.”
BackgroundResultInfluentialSelf-citation
Metadata
- Authors
- Mökander, J., Axente, M., Casolari, F., & Floridi, L.
- Affiliations
- University of Oxford (GB)
- University of Bologna (IT)
- Corresponding author
- Mökander, Jakob
- Venue
- Minds and Machines
- Published
- 2021-11-05
- Volume
- 32
- Issue
- 2
- Pages
- 241-268
- ISSN
- 0924-6495 (ISSN-L)
- 1572-8641
- PMC
- PMC8569069
- OpenAlex
- W3211982349
- Cited by
- 153
- References
- 50
- Citations by year
- 2026: 39 · 2025: 43 · 2024: 23 · 2023: 34 · 2022: 13 · 2021: 1
- FWCI
- 115.46
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Minds and Machines (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- astrazeneca schweiz
- Topics
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
- Keywords
- EU AI Act, AI regulation, artificial intelligence in auditing, conformity assessment, post-market monitoring, high-risk AI systems, internal audit, institutional safeguards, regulatory compliance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.